Microsoft PowerPoint - VCL_Tutorial

March 22, 2007/v1 1

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

Virtual Computing

Laboratory

Aaron Peeler, Josh Thompson, Dr. Mladen Vouk

North Carolina State University

May 07, 2007

March 22, 2007/v1 2

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

Virtual Computing Lab

{ What is it?

z An environment delivery service

z Remote access to high-end software

{ Motivation

z Student Owned Computing

z Distance Education

z Traditional Lab Model

z Shared Compute Resources

z Custom Compute Environments

z Continuous Cycles

March 22, 2007/v1 3

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

What is VCL.

Consumer/Personal Computing

Super

Computing

Services delivered via highly

uniform, extensible, malleable,

& supportable hardware

architecture

Grand challenge class problems

require jump to true super computing

Comprehensive range of

complementary

Services

VCL delivers entire middle tier of

solutions. As such it represents a

transformation in the business

of education—a transformation

in how education is delivered

Universal

Computing Architecture

Productivity Computing

(commodity)

Industrial Computing

(professional)

Enterprise Computing

(business)

High Performance

Computing

(research)

The Computing Pyramid

March 22, 2007/v1 4

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

Motivation - Student Owned Computing

{ University expectation / requirement

{ Stats - NCSU College of Engineering incoming

Freshman

z Fall 2005

{ 81.21% brought laptops

{ 98.39% brought a computer

z Fall 2006

{ 92.99% brought laptops

{ 98.48% brought a computer

March 22, 2007/v1 5

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

Motivation - Distance Education

{ Distance education - how do we get software to

the DE student?

{ Two plus Two program

z Community Colleges

{ Lenoir Community College

{ Craven County Community College

z Cross Institutional curriculums -

{ UNC-CH - Bio Medical Engineering

{ UNCA - Mechatronics Engineering

{ Software restrictions - vendor licensing, must

run on university owned hardware. Violation to

distribute

March 22, 2007/v1 6

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

Motivation - Traditional Labs

Traditional “On campus” computing labs are

important, but are they changing ...

{ Traditional Labs

z Rows & rows of machines

z Single image fits all -- 60+ apps

z Lack of multiple SW versions

z Long time to include new applications

{ Collaborative Learning Spaces

z Flexible - convert the meet instructor demand

z Make use of the user-owned hardware

{ Get more cycles after labs hours

z Provision idle lab machines for remote use.

March 22, 2007/v1 7

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

Motivation - Shared Resources / Custom Env.

{ Shared Resources - Unix “dial-up” servers or

Windows Terminal Server farms

z Ok -- for limited set of users performing limited actions.

z Problems occurs for - special projects or users running

intensive applications. Lack of root/admin access. Just not

scalable!

{ Custom Environments

z Previously -- could not create unique environments in

traditional labs or they could not be supported centrally,

ie. Linux Apache,PHP server for a CSC web development

course - with root access. Through VCL the user can

create these environments which are saved as images and

can be recalled at a later date.

March 22, 2007/v1 8

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

Motivation - Continuous cycles

{ Creative ways to increase the compute cycle of a

single piece of hardware, while meeting the needs of

both researchers and students.

{ Multi-purpose hardware - on-demand use,

dedicated use. Bare-metal,

hyperviser(VMware,KVM,Xen,etc)

{ Switching Blades from VCL to HPC use

z In same Data Center(s)

z Research and Academic computing are out of phase.

March 22, 2007/v1 9

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

http://vcl.ncsu.edu

Users

Database

Management node N

Management node 1

Management node 2

University Labs

Bladecenter

Image Library Image Library

VCL Infrastructure

March 22, 2007/v1 10

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure

{ Web server - Linux Host, RHEL

z Apache

z PHP

{ Database - Linux Host, RHEL

z MYSQL 5.1

{ Management node - Linux Host, RHEL

z xCAT

{ DHCP, tftp, PXE, kickstart install server, all

run on private network

z VCLD - perl, calls xCAT commands, process

assigned requests

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 11

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - LAMP,AJAX

{ Scheduler - based off user’s request provides

optimal hardware. Aware of what is running

where. Assigns request to a Management node

{ Management interface

z Assign user rights - what they can use and do

{ Simply check out environments

{ Manage Computers

{ Manage Images

{ Manage Management nodes

{ Manage User and Resource Groups

{ Modify User Privileges

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 12

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

z Manage Computers

{ Add / delete computers

{ Change attributes about individual computers

z Change states - available, maintenance

z Change Schedule

{ Computer grouping - change groups individual

computers belong to.

{ Reload individual or groups of machines

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 13

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

z Manage Schedules - used to set times

computers are available for use.

{ Add / delete schedules

{ Modify schedules

{ Change ownership of schedules

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 14

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

z Manage Images

{ Create - method to create new images

{ Image grouping - put images in specific image

groups

{ Image Mapping - map image groups to

computer groups

{ Edit image details - rename, change owner,

change minimum requirements (cpu,memory)

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 15

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

z Manage Groups

{ User groups

z Create groups

z Modify groups - members, owners, names

{ Resource groups

z Image groups - groups for images, owner, user

groups that can modify or add images to.

z Computer groups - groups for computers, owner, user

groups that can modify

z Management node groups -create

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 16

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

z Management Nodes

{ Edit Management Node info

z Hostname, IP address, owner, state

{ Edit Management Node Grouping

{ Edit Management Node Mapping

z Map computer groups the management node can

control, reload, provision, etc.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 17

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

z Block Reservations - provision blocks of

computes preloaded for workshops or classes.

{ Set schedule to provision X number of computers

with a particular environment.

{ Repeating schedule, i.e. every Tu,Th 2-4pm

{ List of dates and duration.

z 30-45 minutes before requested start time, the

management node scans all available resources

and starts reload process. All selected resources

are locked out for the group listed in Block

request.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 18

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

{ User preferences - set various prefs.

z Personal Information - set information not pulled

from LDAP, preferred, email or IM notifications,etc

z RDP File Preferences - edit all settings related to

the remote desktop connection, display, mapped

drives, etc.

z View Mode - admin only, change to other users

web view for debugging purposes.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 19

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

{ Privileges - privilege tree

z Based on directory service model

z Set permissions at privilege node for individuals and

user groups

{ Image checkout, Image manage, resource manage,

node manage, user grant, schedule manage

z Set resources available for each node.

z Both permissions and resources can be cascaded to

sub nodes.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 20

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Web portal

Web server - Management Interface(cont’d)

{ Resource Grouping and Mapping

z Resources managed by the groups they are in

z Resource groups of different types are mapped

together

Example:

z Image Groups A & B

z Computer Groups A & B

z Image Group A mapped to Comp. Group A

z Image Group B mapped to Comp. Group B

Image

Group A

Image

Group B

Comp.

Group A

Comp.

Group B

Now, only images in Group A can be run on the computers

that are in comp. Group A. If all of the computers in comp.

Group B are in use and someone requests an image in

image Group B, user must wait until a computer in comp.

Group B is available.

March 22, 2007/v1 21

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Database

Database - Linux, MySQL 5.X

{ Stores all data 46 tables - few examples

z Admin levels - determines what users see

z BlockRequest - workshop course

z Changelog - log of changes to requests

z Computer - info on machines

z Image - info on images

z Imagerevision - revision control

z Managementnode - info, lastcheckin

z Subimages - for cluster based reservations

z Request - core table checked by Management node

z Resourcepriv - privs associated with resource grps

z User - user information

z Usergroup - groups users can be in, custom or LDAP

z Userpriv - privs associated with user groups

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 22

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Management node

Management Node (MN)

{ State driven, polls database for assigned tasks

z New - new request, checks assigned blade, confirms or loads

image, bare metal or VM

z Reserved - waits for user ACK, moves to inuse/timeout

z Inuse - monitors user connection, notifies user of end time, if

user disconnected for given time period, reclaim

z Timeout - based off laststate, reload or mark available

z Deleted - user initiated delete can be at anytime, reload or

make available depending on laststate

z Imageprep - prep environment for imaging mode

z Image - image the machine, partimage for Bare metal,

transfer VM files for hyperviser to Image Library

z MakeProduction - sets images into production based on

image owners request

z Preload - preloads blades for future reservations

z Block Request - prepares blocks of machines for

course/workshops

z Reload - reloads a blade through web app can choose to

reload a specific machine

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 23

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure - Management node

Management Node(MN)

{ Loading process flow, the MN performs these basic checks

when provisioning a blade. Based off the image profile, different

sub tasks are preformed.

z Confirms requested image exists in image library

z Confirms assigned blade/resource is under MN control

z Checks if requested image is preloaded on assigned blade

z Loads blade using xCAT commands if bare metal, if VM copies

vm disk files to VM Host server and starts vm guest

z Monitors loading process

z Updates the request state that blade/resource is ready

z Moves to next state

{ Different provisioning techniques can be implemented, just need

to add support for it into VCLD. xCAT preferred.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 24

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL Infrastructure Management node

1 2 3 4 5 6 7 8 9 1011 12 13 14

Bladecenter

Chassis

Internet

VCL

Management

Node

Management Network

Image

Library

o VCL daemon -- vcld

o xCAT

• DHCP - priv network

• PXE

o Image library

o Manages blades over

private network, loads,

performs tasks, monitors.

Web

Interface

Scheduler

Schedule

Management

Node

Servers

Application

Image

Library

Internet

North Carolina State University Virtual Computing Lab Model

User accesses

web interface

and requests

an application

Scheduler finds

a server with the

requested application

or has management

node load requested

application on a server

Server with requested

application is made

available to user

VCL Manager Software

March 22, 2007/v1 26

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

DEMO

March 22, 2007/v1 27

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL - Image Creation

Image Creation

{ Similar reservation process - except for users with higher

privileges

{ Create new image from existing image. All images started from

base template image. Bare metal or VMWare, Windows or Linux

{ Update images - version control v0…vN

{ Images stored in Image Library and are pushed to blade either

bare metal or VM host server for hypervisor

{ Bare Metal images - copied from disk using Partition Image

(partimage) as part of xCAT.

{ VM images - in case of vmware, copy the vmdk files of image to

image library.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 28

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL - Image Creation

Image Creation - process

{ With higher privileges

z Select Manage Images -> Create New Image from the VCL interface.

{ Choose an existing image to use as base template, usually base

image is a No Apps image for either Windows XP,Windows 2003,

or Linux RHEL4

{ Reserve and Connect to the assigned machine, make changes or

add software packages.

{ Ready to image - return the VCL site under Current Reservations

simply click “Create Image”.

{ Options - update or a create new image.

{ Update image - simply creates the next version of the image,

v1,v2,v3…etc

{ Create image - new name, min requirements(cpu,mem,etc) check

for user connections, etc.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 29

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL - Image Creation

Image Creation - process (behind the scenes)

{ Depending on the OS, and type (bare metal or VM) and even the

profile of the base image, different steps take place during the

image creation process.

{ Common checks / tasks

z Confirm naming for version control

z Defrag - Win only

z Copy any first boot scripts - to rename, start any services that

don’t behave well with preparation tools, sysprep.

z xCAT - set any xCAT configurations for node to be imaged

z Sysprep (MS preparation tool) - default, but is optional based

on image profile.

z Based on profile - start sysprep or simply reboot to start

image capture.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

March 22, 2007/v1 30

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

VCL - Image Creation

Image Creation process (Bare Metal and VM)

{ Bare Metal - uses partimage (www.partimage.org) through xCAT

z Process to copy image from bare disk can take upwards of 15-

20 minutes.

z Saves disk image file in compressed format.

z On average image sizes range from 2-4 GB depending on

applications installed.

{ VM Virtual Machine

z Copies VM disk files to the image library through scp. For

example, in case of VMWare the .vmdk files are copied and

renamed to match the new image name.

z Process less than 4-5 minutes.

Web Frontend

Database

Bladecenter

Management

Node

Image Library

Hardware

Blades, servers, desktops, storage

OS:

Apps

Win Linux

Other …

Virtual

Layer

OS: Win Linux

…

Apps

e.g.,

Web

Sphere

e.g.,

Web

Sphere

SOA

RDP,

VNC,

…

e.g.,

VMWare,

XEN,

MSVS2500,..

X-Win

Client

Apps.

Work

Flow

Services

End-User

Access

Vis

Services

Other …

Middleware

e.g. LSF

VCL

Manager

“Application” Image Stack

xCAT

VCL code

IBM TM

WebServer

DataBase

Etc.

Users

“Images”

H/W Resources

Local or distributed

Differentiator: User to Image to Resource Mapping, Management

& Provenance

Simplicity, Flexibility, Reliability

Scalability, Economy

Web

Interface

Scheduler

Schedule

Management

Node

Institution A

Servers

Management

Node

Servers

Server aggregation in shared

VCL Data Center B

Internet

Management

Node

Servers

Fully distributed VCL Data Center A

Application

Image Library

Institution B

Servers

Web

Interface

Scheduler

Schedule

Remotely scheduled

The Virtual Computing Initiative: Anytime, Anywhere

On Demand Computing

March 22, 2007/v1 33

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

vcl.ncsu.edu

Ongoing:

-Tipping pt.

-Usability

-Availability

-Pedagogy

-…

-Currently

scaling to

8000+ users

-N-M-K

March 22, 2007/v1 34

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 35

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 36

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 37

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 38

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 39

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 40

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 41

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering

March 22, 2007/v1 42

NC State University

Information Technology Division

College of Engineering

NC State

College of Engineering