Posted by: kezhong | April 14, 2010

Building a Xen Virtual Guest using LVM on CentOS 5.4

Recently, I have done the test to build a Xen LVM-based guest on CentOS 5.4 according to Building a Xen Virtual Guest Filesystem using Logical Volume Management (LVM). It took me several days to solve the problems. I record the procedure as below.

List information about the volume group
# vgdisplay
  — Volume group —
  VG Name               datavg
  System ID            
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               239.01 GB
  PE Size               4.00 MB
  Total PE              61186
  Alloc PE / Size       2816 / 11.00 GB
  Free  PE / Size       58370 / 228.01 GB
  VG UUID               a2s1Ii-fxW2-GlVT-CrxQ-uE8c-S9Vl-Gifyb0

Creating a Logical Volume for the Xen Guest System
# lvcreate -L 10G -n xenlv3 datavg
  Logical volume “xenlv3” created

# lvdisplay
  — Logical volume —
  LV Name                /dev/datavg/xenlv3
  VG Name                datavg
  LV UUID                KGnHLB-TQIr-ym1M-uOev-m33h-IsK3-IOTvT3
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                10.00 GB
  Current LE             2560
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  – currently set to     256
  Block device           253:2

Creating a Filesystem on the Logical Volume
# mkfs -t ext3 /dev/datavg/xenlv3
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1310720 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Configuring the Swap Partition for the Xen Guest System
# lvcreate -L 1G -n xenswap3 datavg
  Logical volume “xenswap3” created

# mkswap /dev/datavg/xenswap3
Setting up swapspace version 1, size = 1073737 kB

Mounting the Root Filesystem
# mkdir /xen
# mount /dev/datavg/xenlv3 /xen
# ls /xen
lost+found

Copy the files from the host operating system to the Xen Guest System
# cp -ax /{bin,dev,etc,lib,root,sbin,usr,var} /xen
cp: preserving permissions for `/xen/var/run/cups/certs/0′: Operation not supported
cp: preserving ACL for `/xen/var/run/cups/certs/0′: Operation not supported

# mkdir /xen/{home,proc,opt,sys,mnt,tmp}
# chmod 777 /xen/tmp

Creating a Xen Configuration File
# ls /boot/{init*,vm*}
/boot/initrd-2.6.18-164.15.1.el5.img          /boot/vmlinuz-2.6.18-164.15.1.el5
/boot/initrd-2.6.18-164.15.1.el5xen.img     /boot/vmlinuz-2.6.18-164.15.1.el5xen

# vi /etc/xen/xen3.cfg
kernel = “/boot/vmlinuz-2.6.18-164.15.1.el5xen”
ramdisk = “/boot/initrd-2.6.18-164.15.1.el5xen.img”
memory = 512
name = “Xen3″
vif = [ ” ]
dhcp = “dhcp”
disk = [‘phy:datavg/xenlv3,xvda3,w’, ‘phy:datavg/xenswap3,xvdb3,w’]
root = “/dev/xvda3 ro”

Modifying /etc/fstab for the Guest System
# vi /xen/etc/fstab
/dev/xvda3        /                      ext3     defaults,errors=remount-ro          0 0
/dev/xvdb3       none                 swap    sw                                             0 0
devpts              /dev/pts            devpts  gid=5,mode=620                        0 0
proc                  /proc                 proc      defaults                                     0 0

Booting the Guest OS
# umount /xen
# xm create -c xen3.cfg
… …
Creating root device.
Mounting root filesystem.
mount: could not find filesystem ‘/dev/root’
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory

Xen Guest Boot Problems and solutions
From the above, it was able to boot up. Shutdown and destroy the guest, check the “init” file in the ram disk.

# xm list
Name                            ID Mem(MiB) VCPUs State   Time(s)
Domain-0                      0    30588          8          r—–    502.4
Xen3                             62      512         1          -b—-      0.3
# xm shutdown 6
# xm destroy 6

# cp /boot/initrd-2.6.18-164.15.1.el5xen.img /tmp/initrd.gz
# cd /tmp
# gunzip initrd.gz
# mkdir tmp2
# cd tmp2
# cpio -id < ../initrd
13282 blocks
# more init
… …
echo Scanning and configuring dmraid supported devices
resume LABEL=SWAP-sda3
echo Creating root device.
mkrootdev -t ext3 -o defaults,ro sda2
echo Mounting root filesystem.
mount /sysroot
echo Setting up other filesystems.
setuproot
echo Switching to new root and running init.
switchroot

From the “init” file, we can find it use LABEL=SWAP-sda3 as swap and sda2 as the root file system. So we cannot use it, and we need to generate the ram disk by ourselves.
# mount /dev/datavg/xenlv3 /xen
# cd /boot
# mkinitrd –fstab=/xen/etc/fstab initrd-2.6.18-164.15.1.el5xenU3.img 2.6.18-164.15.1.el5xen

Modify the configuration file and boot guest again.

# vi /etc/xen/xen3.cfg
… …
ramdisk = “/boot/initrd-2.6.18-164.15.1.el5xenU3.img”
… …
# umount /xen
# xm create -c xen3.cfg
… …
Creating root device.
Mounting root filesystem.
mount: could not find filesystem ‘/dev/root’
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic – not syncing: Attempted to kill init!

I checked some materials on the net, someone recommended omit the SCSI module, and then I regenerate the ram disk again.

# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0    30588     8 r—–    527.3
Xen3                                      10      512     1 -b—-      0.3
# xm shutdown 10
# xm destroy 10

# mount /dev/datavg/xenlv3 /xen
# mkinitrd -f –fstab=/xen/etc/fstab –omit-scsi-modules –with=xennet –with=xenblk –preload=xenblk initrd-2.6.18-164.15.1.el5xenU3.img 2.6.18-164.15.1.el5xen
# umount /xen
# xm create -c xen3.cfg
… …
Creating root device.
Mounting root filesystem.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Setting up other filesystems.
Setting up new root fs
no fstab.sys, mounting internal defaults
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
exec of init (/sbin/init) failed!!!: No such file or directory
Kernel panic – not syncing: Attempted to kill init!

I built another Xen guest using virt-install tool and compared their structures. I found I missed lib64 and selinux directories. I copied them and boot the guest. The excited thing happened: the guest boot up!

# mount /dev/datavg/xenlv3 /xen
# cp -ax /{lib64,selinux} /xen
# umount /xen
# xm create -c xen3.cfg

Afterwards, I entered the guest and remove the bridge-utils package.
# yum remove bridge-utils-1.1-2 -y
# reboot

References
Building a Xen Virtual Guest Filesystem using Logical Volume Management (LVM)
How to View, Modify and Recreate initrd.img


Responses

  1. thank u for information

  2. was a nice sharing thanks

  3. this blog is really good and thanks for for giving such a nice blog

  4. SALUTATIONS ! ! ! ! ! ! ! ! ! ! !

    Thankxxxxxxxxxx for the POST …..

    WAS in a fix for three days then found this Worked like a CHARM ! ! !

  5. My name is Shane Golden and I’m a Recruiter for CCC Information Services. We are a software development organization and the nation’s leading provider of advanced collision repair and insurance estimating software solutions, workflow tools and enabling technologies to the automotive claims and collision repair industries. Our client base includes more than 350 insurance companies and more than 20,000 repair facilities. We also deliver the most comprehensive, best-in-class industry insight by leveraging data captured from the millions of transactions processed through our network, and the forward-looking, trusted advisor perspective of our people.

    I came across your website today and was impressed with your qualifications. I’m trying to fill a Sr Linux Administrator position within our Chicago office (located in the Merchandise Mart) and was wondering if you might be interested in learning more about this position or if you might know of someone who is currently looking for new career opportunities.

    I’ve included a job description for your review (below my contact information). If this looks like something in which you might be interested, please respond to this email with a word doc copy of your resume as well as a good time for me to call you to discuss further.

    Thanks in advance and I look forward to your reply.

    Shane Golden
    312.593.8080

    Senior Linux Administrator

    Role Description:
    The Senior Linux Administrator will be tasked with leading a migration effort from Solaris 10 to Oracle Enterprise Linux (OEL) who uses his/her knowledge in the primary area of technical expertise and related technical areas to build, test, and maintain Linux servers that meet business requirements and platform standards. A candidate for this role must have operational experience with mission-critical systems. This role may include on-call responsibilities.

    Essential Duties and Responsibilities:
    • Builds, tests, and maintains Linux based hardware and software infrastructure components
    • Documents all policies, procedures and standards around the build and continuing operation of Linux systems
    • Ensures that infrastructure systems are designed, implemented, and supported to meet CCC’s internal and external standards
    • Solves problems related to installation, implementation, maintenance, and operations of Linux and Unix hardware and software, and related infrastructure components
    • Provides monitoring, coaching, and knowledge transfer to less experienced employees
    • Follows the ITIL standard for all Change Requests, Incidents, and Requests for Service, as documented by CCC’s internal change controls.

    Required Skills:
    • 5+ years of Linux/Unix systems administration experience in a 24×7 environment
    • 3+ years of recent experience with Red Hat Enterprise Linux (RHEL) /Oracle Enterprise Linux (OEL) 5.x preferred
    • 2+ years of Solaris administration experience (Solaris 10 preferred)
    • 2+ years’ experience with LVM
    • Ability to provision, install, configure, operate, and maintain systems including the following operating systems, with emphasis on Linux: RHEL OEL Sun Solaris
    • Jumpstart/Kickstart server configuration and maintenance
    • Shell scripting (Ksh, bash, awk, sed, perl)
    • Deep Linux systems experience including diagnosing performance problems related to network issues, and memory/CPU utilization
    • Experience managing servers in large corporate environments, including security, security policy compliance, and exception processes
    • Excellent communication and documentation skills

    Preferred Skills:
    • HP hardware experience, especially ProLiant DL380 G6 and G7
    • Configuration of HP Insight Manager clients for Linux
    • 2+ years’ experience with Oracle Enterprise Linux 5.x
    • 2+ years experience with Xen hypervisor
    • Experience with Oracle VM
    • Experience administering Solaris ZFS and zpools
    • Experience with Solaris 10 Containers

  6. Was stuck for 2 days. This made it work like a charm.

  7. If you need anything from me you can contact at my %URL% website for more information, and I always have some FREE information or downloads.

  8. I’m gone to tell my little brother, that he should also go to see this webpage on regular basis to take updated from most recent reports.

  9. Hi there it’s me, I am also visiting this web site regularly, this site is truly pleasant and the users are in fact sharing pleasant thoughts.

  10. If some one deszires to bee updated with latest technologies therefore he must bee visit this web
    site and be uup tto date daily.

  11. I loved as much as you’ll receive carried out right here. The sketch is tasteful,
    your authored material stylish. nonetheless, you command
    get got an shakiness over that you wish be delivering the following.
    unwell unquestionably come more formerly again since exactly the same
    nearly very often inside case you shield this hike.

  12. You are in reason for point a very good webmaster. Your website launching pace is actually unbelievable. It sort of feels that you are accomplishing any distinctive tip. Additionally, The actual material are usually must-see. you’ve done a wonderful occupation in this make any difference!

  13. Article writing is also a fun, if you be acquainted with afterward you can write otherwise it is complex to write.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: