Service Assurance/Management: Importing VM Image to Amazon EC2 as an Image

The steps documented here are on how to import vmware images into AWS as AMI images.
For importing the vmware images as EC2 instance, follow the documentation on amazon site.

Also this document is command line approach using the AWS and EC2 API.

Follow the documentation link on AWS site

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instances_of_your_vm.html

Figure below details the process and steps

Step 1) Install the AWS CLI.

Step 2) Prepare the VM for import to Amazon EC2.

Step 3) Export the VM from the virtualization environment.

Step 4) Import the VM into Amazon EC2

Step 5) Launch the instance in Amazon EC2

---------------- STEP 1 ---------------------

Download and Install the AWS CLI Tools

(aws-cli & ec2)

# unzip awscli-bundle.zip

Archive: awscli-bundle.zip

# cd awscli-bundle

# ./install -i /usr/local/aws -b /usr/local/bin/aws

Running cmd: /bin/python virtualenv.py --python /bin/python /usr/local/aws

Running cmd: /usr/local/aws/bin/pip install --no-index --find-links file:///opt/software/awscli-bundle/packages awscli-1.10.45.tar.gz

You can now run: /usr/local/bin/aws --version

# /usr/local/bin/aws --version

aws-cli/1.10.45 Python/2.7.5 Linux/3.10.0-327.el7.x86_64 botocore/1.4.35

# aws configure

AWS Access Key ID [None]: <access_key>

AWS Secret Access Key [None]: <secret_access_key>

Default region name [None]: us-east-1

Default output format [None]:

(OR Follow the below method for aws-cli)

For installing pip, check this url

http://docs.aws.amazon.com/cli/latest/userguide/installing.html

[root@vm01 ~]# /usr/local/aws/bin/pip install awscli --ignore-installed six

You are using pip version 7.0.3, however version 8.1.2 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Collecting awscli

/usr/local/aws/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3 /util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not avail able. This prevents urllib3 from configuring SSL appropriately and may cause cer tain SSL connections to fail. For more information, see https://urllib3.readthed ocs.org/en/latest/security.html#insecureplatformwarning.

InsecurePlatformWarning

Downloading awscli-1.10.44-py2.py3-none-any.whl (970kB)

100% |████████████████████████████████| 970kB 93kB/s

Collecting six

Downloading six-1.10.0-py2.py3-none-any.whl

Collecting s3transfer==0.0.1 (from awscli)

Downloading s3transfer-0.0.1-py2.py3-none-any.whl

Collecting colorama<=0.3.3,>=0.2.5 (from awscli)

Downloading colorama-0.3.3.tar.gz

Collecting botocore==1.4.34 (from awscli)

Downloading botocore-1.4.34-py2.py3-none-any.whl (2.4MB)

100% |████████████████████████████████| 2.4MB 55kB/s

Collecting argparse>=1.1 (from awscli)

Downloading argparse-1.4.0-py2.py3-none-any.whl

Collecting rsa<=3.5.0,>=3.1.2 (from awscli)

Downloading rsa-3.4.2-py2.py3-none-any.whl (46kB)

100% |████████████████████████████████| 49kB 3.5MB/s

Collecting docutils>=0.10 (from awscli)

Downloading docutils-0.12.tar.gz (1.6MB)

100% |████████████████████████████████| 1.6MB 143kB/s

Collecting futures<4.0.0,>=2.2.0 (from s3transfer==0.0.1->awscli)

Downloading futures-3.0.5-py2-none-any.whl

Collecting ordereddict==1.1 (from botocore==1.4.34->awscli)

Downloading ordereddict-1.1.tar.gz

Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.4.34->awscli)

Downloading jmespath-0.9.0-py2.py3-none-any.whl

Collecting simplejson==3.3.0 (from botocore==1.4.34->awscli)

Downloading simplejson-3.3.0.tar.gz (67kB)

100% |████████████████████████████████| 69kB 532kB/s

Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.4.34->awscli)

Downloading python_dateutil-2.5.3-py2.py3-none-any.whl (201kB)

100% |████████████████████████████████| 204kB 404kB/s

Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)

Downloading pyasn1-0.1.9-py2.py3-none-any.whl

Building wheels for collected packages: colorama, docutils, ordereddict, simplejson

Running setup.py bdist_wheel for colorama

Stored in directory: /root/.cache/pip/wheels/21/c5/cf/63fb92293f3ad402644ccaf882903cacdb8fe87c80b62c84df

Running setup.py bdist_wheel for docutils

Stored in directory: /root/.cache/pip/wheels/db/de/bd/b99b1e12d321fbc950766c58894c6576b1a73ae3131b29a151

Running setup.py bdist_wheel for ordereddict

Stored in directory: /root/.cache/pip/wheels/cf/2c/b5/a1bfd8848f7861c1588f1a2dfe88c11cf3ab5073ab7af08bc9

Running setup.py bdist_wheel for simplejson

Stored in directory: /root/.cache/pip/wheels/5a/a5/b9/b0c89f0c5c40e2090601173e9b49091d41227c6377020e4e68

Successfully built colorama docutils ordereddict simplejson

Installing collected packages: futures, ordereddict, jmespath, simplejson, six, python-dateutil, docutils, botocore, s3transfer, colorama, argparse, pyasn1, rsa, awscli

Successfully installed argparse-1.3.0 awscli-1.10.44 botocore-1.4.33 colorama-0.3.3 docutils-0.12 futures-3.0.5 jmespath-0.9.0 ordereddict-1.1 pyasn1-0.1.9 python-dateutil-2.5.3 rsa-3.4.2 s3transfer-0.0.1 simplejson-3.3.0 six-1.10.0

[root@vm01 ~]#

For installing ec2 cli tools

wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

mkdir /usr/local/ec2

unzip ec2-api-tools.zip -d /usr/local/ec2

set JAVA_HOME env variable

$ which java

/usr/bin/java

The which java command executed      earlier returns Java's location in the $PATH environment      variable, but in most cases this is a symbolic link.  The        JAVA_HOME environment variable requires the true path to      the binary.

(Linux only) For Linux systems, you can recursively run the file command on the which java output until you find the binary. For example:

file $(which java)
/usr/bin/java: symbolic link to `/etc/alternatives/java'

The /usr/bin/java location is actually a link to /etc/alternatives/java, so you need to run the file command on that location to see whether that is the real binary.

file /etc/alternatives/java

/etc/alternatives/java: symbolic link to `/usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java'

This returns a new location, which is the actual binary. Verify this by running the file command on this location:

file /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
/usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java: ELF 64-bit LSB executable...

This location is the actual binary (notice that it is listed as an executable). The Java home directory is where bin/java lives; in this example, the Java home directory is /usr/lib/jvm/java-7-openjdk-amd64/jre.

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

$JAVA_HOME/bin/java -version

export EC2_HOME=/usr/local/ec2/ec2-api-tools-1.7.0.0

export PATH=$PATH:$EC2_HOME/bin

export AWS_ACCESS_KEY=your-aws-access-key-id

export AWS_SECRET_KEY=your-aws-secret-key

Verify the tool is working

# ec2-describe-regions
REGION  ap-south-1      ec2.ap-south-1.amazonaws.com
REGION  eu-west-1       ec2.eu-west-1.amazonaws.com
REGION  ap-southeast-1  ec2.ap-southeast-1.amazonaws.com
REGION  ap-southeast-2  ec2.ap-southeast-2.amazonaws.com
REGION  eu-central-1    ec2.eu-central-1.amazonaws.com
REGION  ap-northeast-2  ec2.ap-northeast-2.amazonaws.com
REGION  ap-northeast-1  ec2.ap-northeast-1.amazonaws.com
REGION  us-east-1       ec2.us-east-1.amazonaws.com
REGION  sa-east-1       ec2.sa-east-1.amazonaws.com
REGION  us-west-1       ec2.us-west-1.amazonaws.com
REGION  us-west-2       ec2.us-west-2.amazonaws.com

---------------- STEP 2 ---------------------

Prepare the VM for import to Amazon EC2

1) login to your vm and create a user that will be used for remote access

# adduser netcool

2) Install the application required for remote access.

# apt-get install openssh-server sudo

3) Set sudo permissions for the user created in step 1.

# vi /etc/sudoers

add a new line user #User privilege specification section

netcool ALL=(ALL:ALL) ALL

4) Configure the vm to use DHCP

# echo > /etc/network/interfaces

# vi /etc/network/interfaces

add
iface eth0 inet dhcp

# shutdown -h now

------- STEP 3: Export the VM from the virtualization environment. -----------

Use the vmware export tools to generate the OVA or VMDK files.

for ova from vsphere client

File - Export

------- Step 4) Import the VM into Amazon EC2 -----------

Copy the imported ova image file to S3 bucket

Configure IAM and roles

- Create a new role called vmimport
- Select "Amazon EC2" Role Type
- Attach policy "AdministratorAccess"
- Click and edit Trust Relationship tab and modify the policy to looks as below.
Change ServiceName to vmie.amazonaws.com

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "vmie.amazonaws.com"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "vmimport"
        }
      }
    }
  ]
}

Run the following command to import the image into AWS.

aws ec2 import-image --description "Grafana Linux OVA Image" --disk-containers file://containers.json

containers.json has the options defined. What is the format of the file, what is S3 Bucket Name, What is the S3 file name.

[{
    "Description": "Grafana Image",
    "Format": "ova",
    "UserBucket": {
        "S3Bucket": "grafanaimage",
        "S3Key": "Public_Grafana.ova"
    }
}]

Use the below command to check the status of the image conversion.

aws ec2 describe-import-image-tasks --import-task-ids "import-ami-fgdn6shf"

Once the image is converted, the AMI image for AWS can be found under EC2 Service Dashboard. Select Images->AMIs to see your converted image.

ec2-import-instance "<path-to-vm-image>" -t <instance type> -f <format> -a <architecture> -b <s3 bucket> -o <Access Key ID> -w <Secret Access Key>

 # ec2-import-instance Predict-disk1.vmdk -f VMDK -t m3.xlarge -a x86_64 -b predictive-insights-files -o AKIAJLSZNLHQXXOWWNEA -w Rr4RQsqGoLgdhvXKRk2651zQ0ImkVQRFvb926qJ4 -p Linux

Linux is not supported on the requested instance

Cause: Linux import is only supported on specific instance types. You attempted to import an unsupported instance type.
Resolution: Retry using one of the supported instance types. Microsoft Windows BYOL instances must be launched as a dedicated instances or dedicated hosts for Microsoft Windows, and therefore cannot use the t2 instance type because it doesn't support dedicated instances.

General purpose: t2.micro | t2.small | t2.medium | m3.medium | m3.large | m3.xlarge | m3.2xlarge
Compute optimized: c3.large | c3.xlarge | c3.2xlarge | c3.4xlarge | C3.8xlarge | cc1.4xlarge
Memory optimized: r3.large | r3.xlarge | r3.2xlarge | r3.4xlarge | r3.8xlarge
Storage optimized: i2.xlarge | i2.2xlarge | i2.4xlarge | i2.8xlarge

 For troubleshooting the vm import/export, follow this link

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/VMImportTroubleshooting.html#LinuxNotSupported

Service Assurance/Management

Wednesday, July 6, 2016

Importing VM Image to Amazon EC2 as an Image

Linux is not supported on the requested instance

3 comments: