Google Cloud Instance Configuration – CS [4|5]113, Fall 2021

Note. This tutorial goes through how to set up your own Google Compute Engine (GCE) instance to work on the assignments. Each student will get some credit for use during the semester. Please use the resources judiciously. Adding additional credits is a bit difficult. When you sign up for GCP the first time, you also receive some credits from Google that you can use for the course.

You will connect to the Google cloud through a terminal or shell. This is the standard way of interacting with *nix family systems. This tutorial will also help you set up your own terminal access.

Secure SHell (SSH)

The Secure SHell (SSH) provides a range of secure access tools to remote machines. For this class, we will be using it to establish a terminal (command-line) connection to your virtual machine instance.

We are using key-based authentication to your compute instances. This means that access will be linked to specific computers and accounts that you will be accessing your instance from. Also, you will not use a password for access (unless your local private key is encrypted).

Configuring SSH

Installation

Windows users can look into PuTTY and its documentation. However, I suggest that you set-up your Windows Subsystem for Linux. Doing so will allow you to follow the same (and simplified?) instructions as the Macintosh and GNU/Linux users since most distributions of the latter mentions come with SSH already installed.

Configuration

If you already have a ~/.ssh/id_rsa.pub in your home (user) directory, then you are done. You can check for this file by typing:

machine@user:~$ ls ~/.ssh/id_rsa.pub

You may also create a key pair by running ssh-keygen. It is okay to use an empty passphrase, but doing so means that your private key is unencrypted (this is often okay, since it is stored on your local machine only, but is a problem if your laptop is compromised).

Google Cloud Account

Create a Gmail account if you don’t already have one. You may at https://accounts.google.com. Sign into your account and point your web-browser to https://console.cloud.google.com/education. Fill out the required fields and read the terms and conditions. If you agree, go ahead and click on the blue ACCEPT AND CONTINUE button.

You may receive a verification email, follow the instructions within. At the end you can verify your credits by going to the billing menu (make sure you select the right billing account) as seen in the screenshot bellow.

Cloud Shell

You can access your virtual machine by a terminal that can be reached by pointing your web-browser to https://console.cloud.google.com (make sure you are logged in). You’ll land on a dash-board page. Here you can customise your project (you need one to work on a virtual machine (VM)) or create a new one for the course.

Make sure that the billing account you redeemed your credits on is the one selected for the project as seen in the screenshot above. This is the default setting, but if you already had projects like I did from when I took the course, you’ll need to swap the billing account to the one you just redeemed credit towards.

Now click on the little terminal icon towards the top right corner, your screen will split in half et voila! you have a working shell instance.

Creating or Configuring your Virtual Machine Instance

Create your Instance

1. Navigate to Top left hamburger menu>Compute Engine>VM instances.

1. Click on the blue CREATE INSTANCE button.

You will mostly use the defaults, but change:

• Machine type: micro (this is changeable later)
• Boot disk: select “Ubuntu 21.04” (Or, whatever is the latest release).
• Firewall: Allow HTTP and HTTPS traffic
• Region: us-central1-c
• All full access to all Cloud APIs

Note that the f1-mico (0.6 GB 1 shared vCPU) instance will be free–it will not cost more than the free tier. There are several zone options, so you can choose the one where you would like your virtual instance to live. The closest zone is us-central1, this zone is located in Council Bluffs, Iowa. Set your zone to us-central1-c they will have the configurations we need.

Click CREATE button.

1. After the instance is created: Select the instance from the INSTANCES list and stop it if it is running. You can view your instances here: https://console.cloud.google.com/compute/instances

Reserving an External, Static IP

1. Click on the hyper-linked name of your VM instance: “micro” from the INSTANCES list. This should load another page. Now click on EDIT next to the “VM instance details” heading towards the top.

1. Scroll down to the “Network interface” menu. In the “External IP” field select “Create IP address”. Fill out the fields and click the RESERVE button.

1. Click “Done”. Then scroll all the way down and click the blue SAVE button.

Configure SSH Keys

1. On the same edit page as discussed before, scroll down to the “SSH keys” section and click on the hyperlink “Show and edit”.

2. You may run cat ~/.ssh/name.pub to print your public key to the terminal. Additionally, you may run xclip -sel clip < ~/.ssh/name.pub to copy your public key to the clipboard.

3. Paste the contents into the Enter public SSH key field. Make sure you do not paste any additional new-line or other white-space characters.

Connecting to your Virtual Machine

There are two ways to connect to your Google Cloud VM. We can use their browser interface or your own computer.

Browser Interface

1. Navigate to INSTANCES list page and start the machine we just created and set-up.

1. Now click on the SSH menu under the “Connect” column and then select the “Open in browser window” option from the drop-down.

A pop-up window should appear with a terminal into your VM instance.

Your Personal Machine

1. Open a GNU/Linux shell and run ssh -i PATH_TO_PRIVATE_KEY USERNAME@EXTERNAL_IP

• PATH_TO_PRIVATE_KEY is likely something like ~/.ssh/name.
• USERNAME is the username specified in your public key. Note that this is not your Google username. E. g., your TA’s Google username is ahmadtashfeen46 but the username in his public key is simurgh9.
• EXTERNAL_IP is the IP given in the INSTANCES list under the “External IP” column.

1. Enter the passphrase if prompted.

Configure your Virtual Machine

In order to update and install necessary packages, you may consider running the following commands,

sudo apt update
sudo apt upgrade
sudo apt install -y emacs vim htop tmux tree ranger glances
sudo apt install -y gcc gdb make
sudo apt install -y valgrind strace
sudo apt install -y linux-tools-common linux-tools-generic
sudo apt install -y linux-tools-`uname -r`
sudo apt install -y libcap-dev
sudo apt install -y libacl1-dev build-essential libffi-dev
sudo apt install -y bats zlib1g-dev zlib1g-dbg 
sudo apt install -y libsqlite3-dev sqlite3 bzip2 libbz2-dev 
sudo apt install -y libssl-dev openssl libgdbm-dev libgdbm-compat-dev
sudo apt install -y liblzma-dev libreadline-dev libncursesw5-dev uuid-dev


# Python
sudo apt install -y python3 ipython3 python3-pip
sudo apt install -y python3-tk ssh evince
pip3 install --upgrade pip
python3 -m pip install --user pip pandas numpy matplotlib

# Install pyenv
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
 

For python management you can install pyenv on your systems using the following lines from the docs.

curl https://pyenv.run | bash
echo -e 'if shopt -q login_shell; then' \
      '\n  export PYENV_ROOT="$HOME/.pyenv"' \
      '\n  export PATH="$PYENV_ROOT/bin:$PATH"' \
      '\n  eval "$(pyenv init --path)"' \
      '\n  eval "$(pyenv virtualenv-init -)"' \
      '\nfi' >> ~/.bashrc
echo -e 'if [ -z "$BASH_VERSION" ]; then'\
      '\n  export PYENV_ROOT="$HOME/.pyenv"'\
      '\n  export PATH="$PYENV_ROOT/bin:$PATH"'\
      '\n  eval "$(pyenv init --path)"'\
      '\n  eval "$(pyenv virtualenv-init -)"' \
      '\nfi' >>~/.profile

You should also install docker and make yourself a

curl https://get.docker.com/ | bash
#systemctl enable --now docker # Not needed
sudo groupadd docker
sudo usermod -aG docker $USER
docker --version
systemctl status docker

You might want to reboot now with sudo reboot now.

After a restart, if you followed the instructions above including installing pyenv, you will be able to install docker-compose.

pip install docker-compose

Remarks

For any help about how to set-up a VM instance or connect to it, send the TA an email. For help regarding coupon codes or information that gets you Google credits, email the professor.