使用AWS訓練DNN步驟

發表於 | 分類於 | [object Object]

AWS Instance注意事項及連線

建立AWS instance的時候, 由於我們使用jupyter需要port 8888, 需要 Configure the Security Group

Running and accessing a Jupyter notebook from AWS requires special configurations.

Most of these configurations are already set up on the udacity-carnd AMI. However, you must also configure the security group correctly when you launch the instance.

By default, AWS restricts access to most ports on an EC2 instance. In order to access the Jupyter notebook, you must configure the AWS Security Group to allow access to port 8888.
Click on “Edit security groups”.
On the “Configure Security Group” page:
Select “Create a new security group”
Set the “Security group name” (i.e. “Jupyter”)
Click “Add Rule”
Set a “Custom TCP Rule”
Set the “Port Range” to “8888”
Select “Anywhere” as the “Source”
Click “Review and Launch” (again)

成功建立AWS instance之後, 開啟git bash
ssh -i ‘C:\Users\bobon\.ssh\MyKeyPair.pem’ carnd@54.65.11.64
其中54.65.11.64是instance的ip

AWS上開啟jupyter notebook kernel

首先先把project clone下來, 並設定好conda env

1
2
3
4
git clone https://github.com/udacity/CarND-Traffic-Sign-Classifier-Project
cd CarND-Traffic-Sign-Classifier-Project
conda env create -f environment.yml
source activate CarND-Traffic-Sign-Classifier-Project

接著安裝tensorflow-gpu

1
pip install tensorflow-gpu

opencv 安裝

1
conda install -c https://conda.binstar.org/menpo opencv

剛剛已經建立conda的環境, 且activate CarND-Traffic-Sign-Classifier-Project, 所以可以直接開啟kernel

1
jupyter notebook

在local瀏覽器上輸入http://[all ip addresses on your system]:8888/
例如aws ip為54.65.11.64

1
http://54.65.11.64:8888/

抓取AWS上的資料下來local端

在自己local的terminal上

1
2
scp -i 'C:\Users\bobon\.ssh\MyKeyPair.pem' carnd@54.65.11.64:/home/carnd/Traffic-sign/cnn-traffic-sign* ./models/
scp -i 'C:\Users\bobon\.ssh\MyKeyPair.pem' carnd@54.65.11.64:/home/carnd/Traffic-sign/checkpoint ./models/

接著輸入密碼即可 (carnd)