Google Cloud Instance Configuration – CS [4|5]113, Fall 2022

Note. This tutorial goes through how to set up your own Google Compute Engine (GCE) instance to work on the assignments. Each student will get a limited amount of Google Cloud Education Credits for use during the semester. Please use the resources judiciously. Adding additional credits is a bit difficult. When you sign up for GCP the first time, you also receive some credits from Google that you can use for the course.

You will connect to the Google cloud through a terminal or shell. This is the standard way of interacting with *nix family systems. This tutorial will also help you set up your own terminal access.

Secure SHell (SSH)

The Secure SHell (SSH) provides a range of secure access tools to remote machines. For this class, we will be using it to establish a terminal (command-line) connection to your virtual machine instance.

We are using key-based authentication to your compute instances. This means that access will be linked to specific computers and accounts that you will be accessing your instance from. Also, you will not use a password for access (unless your local private key is encrypted).

Configuring SSH

Installation

Windows users can look into PuTTY and its documentation. However, I suggest that you set-up your Windows Subsystem for Linux. Doing so will allow you to follow the same instructions as the Macintosh and GNU/Linux users since most distributions of the latter mentions come with SSH already installed.

Configuration

If you already have a public key such as ~/.ssh/id_rsa.pub in your home (user) directory, then you are done. You can check for this file by typing:

machine@user:~$ ls ~/.ssh/id_rsa.pub

You may also create a key pair by running ssh-keygen. No file name is needed for generation. It is okay to use an empty passphrase, but doing so means that your private key is unencrypted (this is often okay, since it is stored on your local machine only, but is a problem if your laptop is compromised).

Google Cloud Account

Create a Gmail account if you don’t already have one. You may create an account at https://accounts.google.com. Make sure you are signed into the Gmail account you want the credits to be allocated to. You will need to retrieve your GCP Credit Coupon using the link provided from the instructor. You may receive a verification email, follow the instructions within. Once you recieve a coupon code, redirect your web-browser to https://console.cloud.google.com/education and complete the required fields. If you agree, go ahead and click on the blue ACCEPT AND CONTINUE button.

At the end, you can verify your credits by going to the Billing menu (make sure you select the correct billing account) as seen in the screenshot below.

Cloud Shell

You can access your virtual machine by a terminal that can be reached by pointing your web-browser to the Google Cloud Console (make sure you are logged in). You’ll land on a dash-board page. Here you can customize your project (you need one to work on a virtual machine (VM)) or create a new one for the course.

Ensure you select the Gmail account that uses the Cloud Credits. Select Billing Account for Education as the Billing Account for the project as seen in the screenshot above. If you already have projects, you’ll need to swap the billing account to the Billing Account for Education.

To create a new project or select a different project, follow the steps below. Create a new project or change projects by locating the down arrow in the pop up screen.

A new project can be created by selecting the NEW PROJECT option on the pop up screen. You can change project by selecting the project name.

Name your project, ensure your billing account is set to Billing Account for Education, and select create.

Creating or Configuring your Virtual Machine Instance

Create your Instance

1. Navigate to Top left hamburger menu>Compute Engine>VM instances. Alternatively, you can navigate from the dashboard screen, select Create a VM. Ensure you have the correct project selected before continuing. If the Compute Engine API is not enabled, you will be given an option to enable it before continuing.

1. Click on the blue CREATE INSTANCE button.

You will mostly use the defaults, but change:

• Machine type: micro (this is changeable later)
• Boot disk: select “Ubuntu 22.04” with this configuration: x86/64, amd64 jammy image built on 2022-08-23, supports Shielded VM features.
• Firewall: Allow HTTP and HTTPS traffic
• Region: us-central1-c
• All full access to all Cloud APIs

Note that the e2-micro (1 GB 2 shared vCPU) instance will be cheap but likely not free. You can change to a more expensive instance if you would like, but you will need to be diligent about turning it off when not in use to preserve your credits. There are several zone options, so you can choose the one where you would like your virtual instance to live. The closest zone to Norman, Oklahoma is us-central1. This zone is located in Council Bluffs, Iowa. Set your zone to us-central1-c they will have the configurations we need.

**You may need to increase the system size for different tasks. It is important that you monitor your usage. Shutdown the instance when not in use.**

You can also increase the size of disk space that you would like to use. A Standard persistent disk with a size of 30 should be sufficient.

Setting Boot Disk:

• Click on “Change” button. Left side panel appears.
• Set Operating System as Ubuntu (default Debian).
• Set Verison as Ubuntu 22.04.
• Disk Type: Standard Persistent Disk
• Size: 30 GB Press “Select” to save settings and close menu.

Click CREATE button.

1. After the instance is created: Select the instance from the INSTANCES list and stop it if it is running. You can view your instances here: https://console.cloud.google.com/compute/instances

Reserving an External, Static IP

1. Click on the hyper-linked name of your VM instance: “micro” from the INSTANCES list. This should load another page. Now click on EDIT next to the “Edit micro instance” heading towards the top.

1. Scroll down to the “Network interface” menu. Select the down arrow by the default network settings. In the “External IPv4 address” field, press the down arrow to “Create IP address”. Fill out the fields and click the RESERVE button.

1. Locate the blue SAVE button.

Configure SSH Keys

1. On the same edit page as discussed above, scroll down to the “SSH keys” section and click on the hyperlink “Show and edit”.

2. On your local machine, you may run cat ~/.ssh/id_rsa.pub to print your public key to the terminal. Additionally, you may run xclip -sel clip < ~/.ssh/id_rsa.pub to copy your public key to the clipboard.

3. Select +Add Item. Paste the contents into the SSH key field. Make sure you do not paste any additional new-line or other white-space characters.

4. Locate the blue SAVE button.

Connecting to your Virtual Machine

There are two ways to connect to your Google Cloud VM. We can use their browser interface or your own computer.

Browser Interface

1. Navigate to VM INSTANCES list page and start the desired VM.

1. Now click on the SSH menu under the “Connect” column and then select the “Open in browser window” option from the drop-down.

A pop-up window should appear with a terminal into your VM instance.

Your Personal Machine

1. Open a GNU/Linux shell and run ssh -i PATH_TO_PRIVATE_KEY USERNAME@EXTERNAL_IP

• PATH_TO_PRIVATE_KEY is likely something like ~/.ssh/id_rsa..
• USERNAME is the username specified in your public key. Note that this is not your Google username. E. g., a Google username is jasdehart but the username in their public key is jdehart.
• EXTERNAL_IP is the IP given in the VM INSTANCES list under the “External IP” column.

Example command: ssh -i ~/.ssh/id_rsa jdehart@EXTERNAL_IP

If asked Are you sure you want to continue connecting (yes/no/[fingerprint])? Type yes in the terminal to connect.

1. Enter the passphrase if prompted.

Configure your Virtual Machine

In order to update and install necessary packages, you may consider running the following commands,

sudo apt update
sudo apt -y upgrade
sudo apt install -y emacs vim htop tmux tree ranger glances
sudo apt install -y gcc gdb make
sudo apt install -y valgrind strace
sudo apt install -y linux-tools-common linux-tools-generic
sudo apt install -y linux-tools-`uname -r`
sudo apt install -y libcap-dev
sudo apt install -y libacl1-dev build-essential libffi-dev
sudo apt install -y bats zlib1g-dev zlib1g-dbg 
sudo apt install -y libsqlite3-dev sqlite3 bzip2 libbz2-dev 
sudo apt install -y libssl-dev openssl libgdbm-dev libgdbm-compat-dev
sudo apt install -y liblzma-dev libreadline-dev libncursesw5-dev uuid-dev
sudo apt install -y ssh evince

You might want to reboot now with sudo reboot now.

You will also need to install Python specific packages. Installing these packages will require you to increase your instance size.

Python

sudo apt install -y python3 ipython3 python3-pip
sudo apt install -y python3-tk ssh evince
pip3 install --upgrade pip
python3 -m pip install --user pip pandas numpy matplotlib

Install pyenv

For python management you can install pyenv on your systems using the following lines from the docs. If this error/warning pops up, don’t worry if it fails; Pyenv will still work normally: pyenv: no such command sh-activate

git clone https://github.com/pyenv/pyenv.git ~/.pyenv
cd ~/.pyenv && src/configure && make -C src

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.profile
echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.profile
echo 'eval "$(pyenv init -)"' >> ~/.profile

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
echo 'eval "$(pyenv init -)"' >> ~/.bash_profile

You might want to reboot now with sudo reboot now.

You can install pvenv into your local project directory. Here are the steps to install a local version of Python.

pyenv install 3.10.1
pyenv global 3.10.1
pip install --upgrade pip

Docker

You may also want to install a docker and docker-compose instance.

curl https://get.docker.com/ | bash
#systemctl enable --now docker # Not needed
sudo groupadd docker
sudo usermod -aG docker $USER
docker --version
systemctl status docker

You might want to reboot now with sudo reboot now.

After a restart, if you followed the instructions above including installing pyenv, you will be able to install docker-compose.

pip install docker-compose

GitHub

To use git on your instance you will have to assign a global username and email.

git config --global user.name "Your Name"
git config --global user.email "YourGithub@Email.com"

To use GitHub version control on your new instance, you cannot rely on your password. You will have to create and use a Personal Access Token through GitHub. You can use this generate token as a password when working with remote instances. You will need this to clone or fetch to and from a private repository.

Remarks

For any help about how to set-up a VM instance or connect to it, send the TA an email. For help regarding coupon codes or information that gets you Google credits, email the professor.