CS 5293 Spring 21

Logo

This is the web page for Text Analytics at the University of Oklahoma.

View the Project on GitHub oudalab/cs5293sp22

Google Cloud Instance Configuration – CS 5293, Spring 2022

This tutorial goes through how to set up your own Google Compute Engine (GCE) instance to work on the assignments. Each student will get some credits for use during the semester. Please use the resources judiciously. Adding additional credits is a bit difficult. When you sign up for GCP the first time, you also receive some credits from Google that you can use for the course.

You will connect to the Google cloud through a terminal or shell. This is the standard way of interacting with *nix family systems. This tutorial will also help you set up your own terminal access.

Secure SHell (SSH)

The Secure SHell (SSH) provides a range of secure access tools to remote machines. For this class, we will be using it to establish a terminal (command-line) connection to your virtual machine instance.

We are using key-based authentication to your compute instances. This means that access will be linked to specific computers and accounts that you will be accessing your instance from. Also, you will not use a password for access (unless your local private key is encrypted).

Configuring SSH

Installation

Windows users can look into PuTTY and its documentation. However, I suggest that you set-up your Windows Subsystem for Linux. Doing so will allow you to follow the same instructions as the Macintosh and GNU/Linux users since most distributions of the latter mentions come with SSH already installed.

Configuration

If you already have a ~/.ssh/id_rsa.pub in your home (user) directory, then you are done. You can check for this file by typing:

machine@user:~$ ls ~/.ssh/id_rsa.pub

You may also create a key pair by running ssh-keygen. It is okay to use an empty passphrase, but doing so means that your private key is unencrypted (this is often okay, since it is stored on your local machine only, but is a problem if your laptop is compromised).

Google Cloud Account

Create a Gmail account if you don’t already have one. You may at https://accounts.google.com. Sign into your account and point your web-browser to https://console.cloud.google.com/education. Fill out the required fields and read the terms and conditions. If you agree, go ahead and click on the blue ACCEPT AND CONTINUE button.

Credits page screenshot.

You may receive a verification email, follow the instructions within. At the end you can verify your credits by going to the billing menu (make sure you select the right billing account) as seen in the screenshot bellow.

Billing verification.

Cloud Shell

You can access your virtual machine by a terminal that can be reached by pointing your web browser to https://console.cloud.google.com (make sure you are logged in). You’ll land on a dashboard page. Here you can customise your project (you need one to work on a virtual machine (VM)) or create a new one for the course.

Make sure that the billing account you redeemed your credits on is the one selected for the project as seen in the screenshot above. This is the default setting, but if you already had projects like I did from when I took the course, you’ll need to swap the billing account to the one you just redeemed credit towards.

Now click on the little terminal icon towards the top right corner, your screen will split in half et voila! you have a working cloud shell. But this is not the virtual machine instance you need. Continue reading for instructions on how to create the virtual machine instance.

Shell screenshot.

Creating or Configuring your Virtual Machine Instance

Create your Instance

  1. Navigate to Top left hamburger menu>Compute Engine>VM instances.

VM instance creation.

  1. Click on the blue CREATE INSTANCE button.

Create VM.

You will mostly use the defaults, but change:

VM template.

Note that the e2-micro (1 GB 2 shared vCPU) instance will be cheap but likely not free. You can change to a more expensive instance if you would like, but you will need to be diligent about turning it off when not in use to preserve your credits. There are several zone options, so you can choose the one where you would like your virtual instance to live. The closest zone to Norman, Oklahoma is us-central1. This zone is located in Council Bluffs, Iowa. Set your zone to us-central1-c they will have the configurations we need.

**You may need to increase the system size for different tasks. It is important that you monitor your usage. Shutdown the instance when not in use.**

You can also increase the size of disk space that you would like to use. A Standard persistent disk with a size of 30 should be sufficient.

Setting Boot Disk:

VM template.

Click CREATE button.

  1. After the instance is created: Select the instance from the INSTANCES list and stop it if it is running. You can view your instances here: https://console.cloud.google.com/compute/instances

VM instance micro.

Reserving an External, Static IP

  1. Click on the hyper-linked name of your VM instance: “micro” from the INSTANCES list. This should load another page. Now click on EDIT next to the “VM instance details” heading towards the top.

Edit VM instance.

  1. Scroll down to the “Network interface” menu. Press the pencil in the top right of the “Network Interface” Box. In the “External IP” field, press the down arrow and select “Create IP address”. Fill out the fields and click the RESERVE button.

Reserve menu.

  1. Then scroll all the way down and click the blue SAVE button.

Configure SSH Keys

  1. On the same edit page as discussed before, scroll down to the “SSH keys” section.

  2. You may run cat ~/.ssh/id_rsa.pub on your local machine to print your public key to the terminal. Additionally, you may run xclip -sel clip < ~/.ssh/id_rsa.pub to copy your public key to the clipboard. (The name of the .pub file may vary.)

  3. Click the ADD ITEM button and paste the contents into the Enter public SSH key field. Make sure you do not paste any additional newline or other whitespace characters.

  4. Scroll down and click the SAVE button.

SSH public key.

Connecting to your Virtual Machine

There are at least two ways to connect to your Google Cloud VM. We can use their browser interface or your own computer.

Browser Interface

  1. Navigate to INSTANCES list page and start the machine we just created and set up.

Start the VM.

  1. Now click on the SSH menu under the “Connect” column and then select the “Open in browser window” option from the dropdown.

SSH in broswer.

A pop-up window should appear with a terminal into your VM instance.

Terminal.

Your Personal Machine

  1. Open a GNU/Linux shell and run ssh -i PATH_TO_PRIVATE_KEY USERNAME@EXTERNAL_IP
  1. Enter the passphrase if prompted.

SSH on personal machine.

Configure your Virtual Machine

In order to update and install necessary packages, you may consider running the following commands,

sudo apt update
sudo apt upgrade -y
sudo apt install -y emacs vim htop tmux tree ranger glances
sudo apt install -y gcc gdb make
sudo apt install -y valgrind strace
sudo apt install -y linux-tools-common linux-tools-generic
sudo apt install -y linux-tools-`uname -r`
sudo apt install -y libcap-dev
sudo apt install -y libacl1-dev build-essential libffi-dev
sudo apt install -y bats zlib1g-dev
sudo apt install -y libsqlite3-dev sqlite3 bzip2 libbz2-dev 
sudo apt install -y libssl-dev openssl libgdbm-dev libgdbm-compat-dev
sudo apt install -y liblzma-dev libreadline-dev libncursesw5-dev uuid-dev
sudo apt install -y ssh evince

You might want to reboot now with sudo reboot now.

You will also need to install Python specific packages. Installing these packages will require you to increase your instance size.

Python

sudo apt install -y python3 ipython3 python3-pip
sudo apt install -y python3-tk ssh evince
pip3 install --upgrade pip
python3 -m pip install --user pip pandas numpy matplotlib

For python management you can install pyenv on your systems using the following lines from the docs.

curl https://pyenv.run | bash
echo -e 'if shopt -q login_shell; then' \
      '\n  export PYENV_ROOT="$HOME/.pyenv"' \
      '\n  export PATH="$PYENV_ROOT/bin:$PATH"' \
      '\n  eval "$(pyenv init --path)"' \
      '\n  eval "$(pyenv virtualenv-init -)"' \
      '\nfi' >> ~/.bashrc
echo -e 'if [ -z "$BASH_VERSION" ]; then'\
      '\n  export PYENV_ROOT="$HOME/.pyenv"'\
      '\n  export PATH="$PYENV_ROOT/bin:$PATH"'\
      '\n  eval "$(pyenv init --path)"'\
      '\n  eval "$(pyenv virtualenv-init -)"' \
      '\nfi' >>~/.profile

You can install pvenv into your local project directory. Here are the steps to install a local version of Python.

pyenv install 3.10.1
pyenv global 3.10.1
pip install --upgrade pip

Pipenv

We will use pipenv to manage our python environment instead of Anaconda.

pip install -U pipenv

Docker

You may also want to install a docker and docker-compose instance.

curl https://get.docker.com/ | bash
#systemctl enable --now docker # Not needed
sudo groupadd docker
sudo usermod -aG docker $USER
docker --version
systemctl status docker

You might want to reboot now with sudo reboot now.

After a restart, if you followed the instructions above including installing pyenv, you will be able to install docker-compose.

pip install docker-compose

GitHub

To use git on your instance you will have to assign a global username and email.

git config --global user.name "Your Name"
git config --global user.email "YourGithub@Email.com"

To use GitHub version control on your new instance, you cannot rely on your password. You will have to create and use a Personal Access Token through GitHub. You can use this generated token as a password when working with remote instances. You will need this to clone or fetch to and from a private repository.

Remarks

For any help about how to set-up a VM instance or connect to it, send the TA an email. For help regarding coupon codes or information that gets you Google credits, email the professor.