zsh

Nice Linux tutorial.

There are many shells one can use, Mac default is zsh (located at /bin/zsh). I set up PyCharm to use it as well in the Preferences\Tools\Terminal\Shell Path

Bash/zsh comment is the same as python: #

# To open Pycharm/VSCode from terminal:
open -na "PyCharm.app" .
open -na "Visual Studio Code.app" .
# To open a finder from terminal:
open .

To open terminal from folder I added a shortcut: Control + Alt + Shift + T

To see hidden files on Mac in finder: Command + Shift + .

Command History

history: To see command history
!10: to call command on line 10
!! to call last command
!string to call last command that starts with string
history -c to clear history

Environment variables

export A="some value"   # set env variable A
env                     # see all variables
unset A                 # remove env variable A

Variable

variable=<some_value>  # make sure there are no spaces around =
variable=solar

To replace chars inside a string:

repo_name="home/solar"
echo $repo_name
repo_name=${repo_name//[\/l]/_}
echo $repo_name

ls command

To count files in a folder from terminal:

ls /etc | wc -l

recursively:

find <directory> -type f | wc -l

for example:

ls '/Users/nenad.bozinovic/work/Frame/elasticity_logs' | head -4

to see file sizes:

ls -l
ls -l --block-size=M

to see size of the whole folder:

du -sh share

to see hidden files:

ls -a
ls -la

Sorts by the last date modified:

ls -lt

goes to previous folder:

cd -

tree command

To see tree of directories use tree command:

tree --help

To install on Linux:

!sudo apt install tree

to install on Mac:

brew install tree

gz files

To zip/unzip gz:

Install gnu-tar if seeing warning with the tar, then use gtar:

brew install gnu-tar

To zip:

gtar -zcvf myfolder.tar.gz myfolder
gzip filename  # zip it back
gzip -k filename  # to zip it and keep original

to see the content of a zipped file:

gtar -tf myfolder.tar.gz

to unzip:

tar -xf labeled_data.tar.gz
tar -xf labeled_data.tar.gz -C /home/user/destination

zip files

zip filename.zip file1.txt file2.txt file3.txt
unzip -vl titanic.zip  # to see content without unzipping
unzip filename.zip
unzip filename.zip -d /home/user/destination

Download

To download a file from URL:

!curl -OL <URL>

From Google drive (FILE_ID can be found in the sharable link between /d and /view):

import gdown
gdown.download("https://drive.google.com/uc?id=FILE_ID", 'NEW_NAME.jpg')

Paths

Print path:

echo "${PATH//:/$'\n'}"

To add folder to PATH

Homebrew

Homebrew is a package manager for macOS, it might have some unique packages that pip doesn’t have, to install wget for example:

brew install wget
wget https://your.link.png

pip

To install package:

pip install torch torchvision tensorboard

When using [] with pip it is important to use "" to avoid shell parsing for example:

pip install "mpl_interactions[jupyter]"

To see version of the package installed:

pip show torch

To see all packages installed:

pip list

vim

Command Explanation
i insert mode
Esc exit insert mode
:w Saves the file you are working on
:w [filename] Allows you to save your file with the name you’ve defined
:wq Save your file and close Vim
:q! Quit without first saving the file you were working on
v visual mode, this allows you to select the text you want to copy
y yank (i.e. copy)
p paste

if statement in zsh

if ! [[ -n $repo_name ]] || ! [[ -n $python_ver ]]; then  
  echo "you didn't enter repo name and/or python version"  
  exit 0  
fi

Get IP address

To be able to use ip:

brew install iproute2mac

Before you generate ssh key pair check if you have one:

cat ~/.ssh/id_rsa.pub

SSH

In order to establish ssh connection with other servers (GitHub, Paperspace, etc) one must set exchange ssh public keys.

To generate ssh key pair (rsa type) use:

ssh-keygen

(you should see id_rsa and id_rsa.pub files that relate to private and public keys).

To print public key:

cat ~/.ssh/id_rsa.pub

You can directly copy the key to the remote machine if you know username and the server address (use whoami and hostname to get user@server):

ssh-copy-id -i ~/.ssh/id_rsa.pub user@server

ssh config

To see all remote machines added to local computer:

cat ~/.ssh/config

Alias

To change the prompt when ssh into a machine edit .bashrc file:

vim ~/.bashrc

Look for the line that sets the PS1 variable, which defines the shell prompt. It may look something like this:

PS1="\u@\h \w $ "

The \u represents the username, \h represents the hostname, and \w represents the current working directory.

Modify the PS1 line to include the desired alias instead of the hostname. For example:

PS1="\u@alias_name \w $ "

To apply the changes, either log out and log back in, or run the following command:

source ~/.bashrc

This will reload the shell configuration file and update the shell prompt.

Hardware stats

Use lscpu to get info about CPU installed (or cat /proc/cpuinfo). On Mac (sysctl -n machdep.cpu.brand_string)

Use nvidia-smi to see GPU stats (nvidia-smi -L for GPU full name).

List of basic linux commands

Python version

which python

Gives for example: /opt/conda/bin/python

Copy files/folders

Use scp. For example, to copy file to server:

scp fiename.txt root@ip_address:/home/user
scp fiename.txt root@ip_address:/home/user
scp fiename.txt root@ip_address:~  # into home directory
scp fiename.txt root@ip_address:~/new_name.txt

to copy file from server:

scp root@ip_address:~/filename.txt local_path
scp root@ip_address:~/filename.txt .  # into current directory

For folders add -r:

scp -r test_folder root@ip_address:path_to_folder

To see status of the copying:

rsync -ah --progress /path/to/source_directory/ /path/to/destination_directory/

Rename files/folders

mv source dst # to move and/or rename directory

Delete files/folders

rm file1.txt file2.txt

Delete directory:

rm -r  # delete non empty directory
rmdir # to delete empty directory
rm -d  # to delete empty directory
mkdir folder_name

Username

useradd username
passwd username
login username
whoami

IP Address

To get IP address on Linux, use curl this:

apt install curl
curl ifconfig.me
curl checkip.amazonaws.com

Get system, host, and user names

lsb_release -a        # gives system info like Linux version
uname -a              # gives system info like Linux version
cat /etc/os-release   # gives system info like Linux version
hostname
whoami

To get a hostname in Jupyter:

hn=!hostname
hn[0]
'Jupiter-2.local'

Find shasum

To get a shasum of a file:

shasum -a 256 file1.txt file2.txt ...

To do this via pipe:

find . -name abc.txt | xargs shasum -a 256

Concatenate strings

VAR1="Hello, "
VAR2=2
VAR3=" Worlds"
VAR4="$VAR1$VAR2$VAR3"
echo "$VAR4"

Executable file

chmod +x some_file.ext

Check if there is x in front of the name with:

ls -l

Disk usage

Take from here:

df -h   # to check disk space
du -h   # to check disk usage
du -h <folder>  # specific folder 
du -a <folder> | sort -n -r | head -n 10  # show top 10 files

Install library


sudo apt-get update
sudo apt-get install <package_name>
dpkg -l | grep <package_name>   # check if package is installed

Install drivers for NVIDIA card

As usual first:

sudo apt update
sudo apt upgrade -y

Then find out the GPU driver version:

ubuntu-drivers devices

then install the recommended driver:

sudo apt install nvidia-driver-XYZ

you will need to reboot the system after the installation.

sudo reboot

finally check the driver version:

nvidia-smi