Thursday, May 5, 2011

Create Ubuntu AMI from Scratch on local machine

This guide will explain about creating AMI from scratch. Here we will create AMI on local system. The main benefit of creating AMI on local system is cost saving; we do not need to launch instance for configuring application. Instead we can configure our OS, install / configure required software and then create AMI on local system. Then we can upload newly created AMI on s3. Now from this AMI we can launch instance when we need them. In this way we will get pre-configured instance. In this tutorial we will create Ubuntu AMI from scratch. You can also follow same procedure on cloud (ie you can create this on instance also)

In this tutorial we will create (create AMI from scratch on local system), bundle (bundle the image), upload (upload newly created AMI on s3), run(run the instance based on this AMI) AMI.

What AMI is: An Amazon Machine Image (AMI) is a special type of virtual appliance which is used to instantiate (create) a virtual machine within the Amazon Elastic Compute Cloud. It serves as the basic unit of deployment for services delivered using EC2. We can say that AMI is an image from which an instance can boot


Install required dependencies:
$ apt-get install ssh
$ apt-get install debootstrap
$ apt-get install ruby
$ apt-get install sun-java6-jdk
$ apt-get install libopenssl-ruby
$ apt-get install curl
$ apt-get install

Create disk image:
Firstly we need to create disk image. In this case we will create a disk image of 1000MB
$ dd if=/dev/zero of=ubuntu-ami count=1000 bs=1M 

Make a filesystem on the disk image:
$ mke2fs -F -j Ubuntu-ami 

Mount newly created disk image:
$ mount -o loop ubuntu-ami /mnt

Install ubuntu into /mnt:
$ lsb_release –c
Above cmd will give name of the your distribution (let DISTRIBUTION) (eg: hardy or jaunty or maverick or lucid etc.)
$ debootstrap  DISTRIBUTION /mnt/

Note: In the last step, the DISTRIBUTION you specified will be installed, eg if you specified
lucid then Ubuntu 10.04 LTS will be installed
dapper then Ubuntu 6.06 LTS will be installed
maverick then Ubuntu 10.10 LTS will be installed

Enter chrooted environment:
$ chroot /mnt

Create devices:
$ mount -t proc proc /proc    
$ cd /dev
$ MAKEDEV console
$ MAKEDEV std

Changed root password:
$ passwd

Configure network interface
$ echo -e 'auto lo\niface lo inet loopback\nauto eth0\niface eth0 inet dhcp' >> /etc/network/interfaces

Amend /etc/fstab file (We also need to define some mount points)
echo -e '/dev/sda1 / ext3 defaults 0 1\n/dev/sda2 swap swap defaults 0 0' > /etc/fstab

Install sshd (Once we would have our new AMI ready, uploaded and started we would connect to it via ssh)
$ apt-get install ssh
Note: Now you can install required software by apt-get

You may need to copy sources.list to mounted OS, if host OS and mounted OS are same (both OS as well as version / distribution), then
$ exit
$ cp /etc/apt/sources.list /mnt/etc/apt/sources.list
Same way you can copy any data you want

Exit / umount chroot environment
$ exit
$ umount -l /mnt

Now you are in host OS, Now we will bundle the image and upload it on S3

Install AMI and API tools:
Add repository:
$ sudo perl -pi -e 's%(universe)$%$1 multiverse%'  /etc/apt/sources.list

Update:
$ sudo apt-get update

Install ec2 tools:
$ sudo apt-get install ec2-ami-tools
$ sudo apt-get install ec2-api-tools

Define following environment variable (in .bashrc):
export EC2_PRIVATE_KEY=PATH-TO-PRIVATE-KEY
export EC2_CERT= PATH-TO-X.509-CERTIFICATE

Bundle the image:
$ ec2-bundle-image -i ubuntu-ami --cert PATH-TO-X.509-CERTIFICATE--privatekey PATH-TO-PRIVATE-KEY-u AWS-ACCOUNT-NO

Upload AMI to S3:
$ ec2-upload-bundle -b S3-BUCKET -m /tmp/ubuntu-ami.manifest.xml -a AWS-ACCESS-ID -s AWS-ACCESS-KEY

Register the AMI:
$ ec2-register –K  PATH-TO-PRIVATE-KEY –C PATH-TO-X.509-CERTIFICATE  S3-BUCKET/ubuntu-ami.manifest.xml

Run Instance using newly created AMI:
$ ec2-run-instances AMI-ID -g SECURITY-GROUP -k KEY-PAIR -z ZONE --kernel KERNEL-ID

Note:
AMI-ID: which you got in last step
KERNEL-ID: Cloud market would be right place to find required kernel-id (Find your system info by executing $ uname -a)

  Paths:

        PATH-TO-PRIVATE-KEY:   /mnt/pk*.pem
        PATH-TO-X.509-CERTIFICATE:  /mnt/cert*.pem


  Keys and certificate:
        AWS-ACCOUNT-NO:   In the web browser click “account >> Account Activity” (It will appear on right side in the form 9999-9999-9999. When you use the account number in the context of the API's, you should leave out the hyphens and just use the 12 digits)
        AWS-ACCESS-ID:   In the web browser click “account >>security credentials” under heading “access credentials >> access keys”
        AWS-ACCESS-KEY:   In the web browser click “account >>security credentials” under heading “access credentials >> access keys”
        PRIVATE-KEY:   In the web browser click “account >>security credentials” under heading “access credentials >> x.509 certificate”
        X.509-CERTIFICATE:   In the web browser click “account >>security credentials” under heading “access credentials >> x.509 certificate”



6 comments:

  1. This is so cool. I can't actually believe how well this works. Saves me so much time. Thanks.

    ReplyDelete
  2. I did this but the AMI didn't boot due to the Kernal not being Xen compatible

    ReplyDelete
  3. @Doug thanx
    I updated the instructions and added to specify kernel while launching the instance

    ReplyDelete
  4. prachi agrawalMay 27, 2011

    Thanks for so much help..........

    ReplyDelete
  5. I tried this and when I launch the instance, it never passes the "Instance Status Checks" which monitors the software and network configuration for the instance. I cannot ssh to the machine either.

    So, how does this image know what ssh key to put in a user's authorized_keys?

    ReplyDelete
  6. I have exactly the same problem as "ncc". SSH times out and instance status checks never pass either. What might be up?

    ReplyDelete