Date/Time: Wed, December 7th, 2011 @ 8:00p EST
Register: https://www1.gotomeeting.com/register/945190048

If anyone has attended an event of this nature (discussion forum, Q&A panel), the success of said event is highly dependent upon the interaction and participation of both the audience and the panelists.  Accordingly, there needs to be an abundance of healthy and meaningful discussion topics so that the event does not turn into a 3rd grade staring contest, or even worse, a FUD-flinging competition.  That responsibility is on us – the community.

In an effort to provide scintillating, thought-provoking topics of discussion, the below are my contributions to the forth-coming entertainment.

1. Do you see large enterprises adopting unified platforms such as FlexPod or Vblock in place of their deeply-entrenched and traditional stack infrastructures?  Or is it more of a niche play at the moment?
2. How do you envision the evolution of technologies like intelligent caching algorithms (VST, FAST) and server-side SSD memory caches (Project Lightning) shape how customers consume data?
3. As geographically distributed storage solutions like MetroCluster and VPLEX become more popular in practice, what can you divulge about the future innovations of continuous availability solutions?
4. How do you see VMware’s expansion into providing storage technologies like the vSphere Storage Appliance (VSA) and host-based replication compliment the current SMB storage array offerings?
5. Has there been any progress made around the recent performance issues with the VAAI Thin Provision Block Reclaim primitive (UNMAP)?

There you have it.  My take on vendor-neutral, non-FUD provoking, and intellectually-compelling questions.  None of this “who’s best?” nonsense.

I encourage all who are interested to reach out to Damian Karlson (@sixfootdad) to submit your questions for this event.

UPDATE (12/9): Listen to this recorded BrownBag on Vimeo or iTunes.

Data Loss Statistics (National Archives & Records Administration in Washington)

• 93% of companies that lost their data center for 10 days or more due to a disaster filed for bankruptcy within one year of the disaster
• 50% of businesses that found themselves without data management for this same time period filed for bankruptcy immediately

Recovery Requirements (RPO)

• Days – backup/restore (NetApp snapshots, NetApp SMVI)
• Hours – Asynchronous replication, scripted or manual recovery workflows (NetApp SnapMirror and VMware SRM)
• Minutes – Synchronous replication, continuous availability, geo-clusters and distributed applications (NetApp MetroCluster and VMware HA FT)

Datacenter Availability Components

• Continuous availability to service end users
• Non-disruptive maintenance operations
• Easy to manage infrastructure components
• Integration with all aspects of management

VMware High Availability

• Physically stretch ESX cluster across a campus or metro distance
• HA extended between distributed parts of the same virtual datacenter
• Automatic rapid recovery from host failures
• No complex clustering software required

VMware Fault Tolerance

• Easily enabled on a per VM basis
• Eliminate downtime altogether with secondary VM
• Protect homegrown apps w/o clustering solution

VMware Host Affinity

• Provides site affinity capabilities
• Keeps workloads local to home storage
• Keeps primary and secondary FT VMs at appropriate sites

Stretching VMware vSphere and NetApp

• MetroCluster = Replicated storage device at each site
  *Single FAS HA pair split across two geographic locations (Controller 1 at Site A / Controller 2 at Site B)
• Virtual machines housed inside of a LUN/volume, contained within an aggregate
• Underneath the aggregate is a plex (half of a synchronously mirrored aggregate)
• Array based synchronous replication (SyncMirror) replicates plexes (think RAID 1+0)
• Aggregate-level synchronous replication = any action taken within the aggregate is mirrored to the respective aggregate (within the remote plex)
• RAID-DP raid groups + mirroring to remote aggregate

MetroCluster Use Case = Planned datacenter migration (moving operations between campus/metro locations)

1. vMotion VMs to recovery site
2. Perform storage takeover from recovery site (changes point of access for storage – plex at remote site becomes active plex used for read/write operations)
3. Shutdown recovery site for maintenance or planned disaster
4. Power on primary site (automatic resync of storage plexes)
5. vMotion VMs back to primary site
6. Perform storage takeover from primary site

vCenter MetroCluster Plugin

• VI administrator has complete oversight and ability to drive virtual infrastructure and storage operations from SPOG (vCenter)
• Define local/remote site of MetroCluster
• Detects Datastores, VMs, Hosts

1. Evacuate all VMs = puts ESX hosts in maintenance mode, automatically vMotions all VMs to remote site
2. On remote storage controller, execute takeover (normal failover = non-disruptive)

Continuous Availability and DR

• Dedupe replication at array-level (eliminates full, hydrated datasets across wire)
• Native WAN compression (only sends changed blocks)
• VMware vSphere 5 integration (non-disruptive testing and automated failback)
• Global disaster recovery = Protected site (metrocluster) asynchronously replicated with SRM to a third recovery site (geographically unlimited distance)

Session Questions

1. Is there still a requirement for backups if replicating to remote site?
   Scott: You can replicate data to the other side (D2D), and at the replication site you can do backups.
   Larry: There are customers using SnapMirror as a backup tool, archive or long-term solution = vaulting solution, you can perform backups of data after replicated to recovery site.
   From both speakers, and as is usually always the case, it depends on business requirements.
2. From the SnapMirror perspective, are backups incremental?
   Scott: The value of SnapMirror is that you control how often they are incremental.  Full backups can be scheduled with incrementals thereafter.
   Larry: SnapMirror is snapshot replication technology, and you can revert back to a snapshot (entire copy of filesystem)
3. How are you determining placement of VMs?
   Larry: Placement of VMs determined initially from the ESX host when first configuring plugin.  NetApp is reviewing results with VMware for vSphere 4.1 and host affinity, plugin will validate host affinity groups and determine site membership based on that.
4. Are hosts still reading from primary storage location after vMotion to recovery site?
   Larry: Yes, until storage takeover is completed.
   Scott: Question touched on two main points of the presentation: Ease of management and continuous access to backend (nice segue to bring it full circle to the earlier-discussed 5 tenets of datacenter availability)

My personal editorial comments:

Presentation Quality and Speaker Assessments

• Extremely valuable and well-designed technical content (real-time demos are always helpful)
• Well-prepared and well-spoken presenters (great recovery in light of technical difficulties; Public speaking rule #1: prepare for the worst; Public speaking rule #2: make a joke about it – well played)
• I dig the idea of opening the floor for 15 minutes of Q&A at the end (especially for a highly-misunderstood and complex topic such as geographic clustering solutions)
• Nailed perhaps the most underrated component of speaking in front of large audiences that may not be able to hear all of the questions (especially for recorded sessions)… presenters repeating the questions is crucial!  Well done.
• One suggestion (and I’m stretching here) for the presentation demo would be to use DNS names for the OAK and SFO ESX hosts to visually demonstrate to the audience the VM locality before and after failover (i.e. this is where the VMs live now, this is where they live after – Larry did mention this, but may have been easier for the audience to conceptualize with DNS instead of IP)

Now for the technical stuff…
For disaster avoidance, NetApp MetroCluster with vSphere HA/DRS/FT is a distinct and unparalleled solution that offers non-disruptive workoad mobility.

For disaster recovery, NetApp MetroCluster with vSphere HA/DRS/FT provides an automated solution for a wide variety of failure scenarios.
*Depending on the disaster, additional steps may need to be taken to recover VMs.  Huh?

During an entire site failure, an automated cluster takeover will not be initiated by the surviving storage controller node.  This behavior is, expectedly so, to mitigate the potential risk of split brain scenarios where the controller node at the remote site “unknowingly” performs a storage takeover.  Therefore, manual intervention is required!

1. Perform force takeover on surviving controller [cf forcetakeover -d]
2. Register VMs in vCenter
3. Power on VMs

It’s not uncommon for an organization to take some valuable time in evaluating a complete site failure and declare a disaster.  Specific change procedures, personnel organization, facilities recovery, and sign-off by C-level management is oftentimes required before any failover actions can be instantiated.  The caveat here is that if Step 1 is not completed before HA has a chance to restart the VMs at the remote site, you will have to manually register and power on your entire virtual machine inventory from the failed site.  Steps 2 and 3 can become increasingly tedious, especially when scaling to large environments.  PowerShell scripts can help ease the administrative burden with this process.  This script, written by the masterful @LucD22, searches for all VMX files in all or specific datastores and registers them in vCenter.

Compared with SRM (which is an entirely different topic in itself), the failback with a MetroCluster solution is easy, efficient, and non-disruptive.  VMware has done an impressive job from 4.1 to 5.0 with enhancing failback capabilities within SRM, however this is still an inherently disruptive process.  With the MetroCluster solution, it’s essentially a storage resynch, vMotion of VMs back to primary site, and issue the storage giveback.  It really is that easy, which makes it a robust, yet simple, solution.

Some thoughts for improvement (VMware/NetApp):

• SRM compatibility with MetroCluster solutions (especially so for bi-directional, active/active DCs without a third colo)
• Enhancements around site affinity (increased operational overhead, not scalable)
• Tight integration with vCenter (MetroCluster vCenter plugin is awesome!)
• Improved workflows and integration with vSphere to handle complete site failures and/or split-brain scenarios
• Official support and certification from VMware and vSphere 5 HCL inclusion

Another thought I had was for a VMworld session which revolves around MetroCluster design considerations and the demonstration of the below failure scenarios and infrastructure impacts (modeled around TR3788 – A Continuous-Availability Solution for VMware and vSphere and NetApp):

• Disk shelf
• Disk loop
• Storage controller
• Storage contoller interconnect
• ESX hosts
• Interconnect switch
• ISLs
• Complete site

@NetApp: I would love to present it. ;]

My notes remain storage agnostic and generally describe the advantages and disadvantages of disaster recovery and avoidance solutions without vendor references.  

Disaster Avoidance != Disaster Recovery
Site-level logic is the same as host-level logic

Host Level

• Disaster avoidance = vMotion to avoid disaster and outage (non-disruptive)
• Disaster recovery = HA restarts VMs (disruptive)

Site Level

• Disaster avoidance = vMotion over distance to avoid disaster and outage (non-disruptive)
• Disaster recovery = SRM or scripted register/power-on of VMs at recovery site (disruptive)

Types of disasters dictate different disaster recovery behaviors
Hurricane (planned = disaster avoidance) vs. earthquake (unplanned = disaster recovery) is materially different

Types of Stretched Cluster Implementations

Single stretched vSphere cluster

• Intra-cluster vMotions are parallelized
• vMotion network requirements = 622Mbps/5ms RTT, L2 equivalence for VMkernel (support requirement) and VM network traffic (operational requirement)

Multiple vSphere clusters

• Inter-cluster vMotions are serialized
• vMotion network requirements = 622Mbps/5ms RTT, L2 equivalence for VMkernel (support requirement) and VM network traffic (operational requirement)

Classic Site Recovery Manager implementation

• 2 vCenters with one at each site managing infrastructure
• Array-based (sync, async, continuous) replication or vSphere Replication
• No single point of failure on management infrastructure
• Read-only storage gets promoted or snapshot’ed to become writeable

Good point here by Chad: what if you’re the smartest person in the room and your organization requires operational simplicity if you’re involved in the disaster?  SRM is an easy push-button mechanism.

SRM and non-disruptive workload mobility is mutually exclusive!

• vMotion = single vCenter domain (non-disruptive workload mobility)
• SRM = two or more domains (disruptive disaster recovery or workload migrations)

vSphere Stretched Cluster Design Considerations

1. HA does not follow a recovery plan workflow = dependencies lead to complex scripting for proper power-on sequencing (precisely what SRM does)
2. Single vCenter protection needs to be part of design process (vCSHB)
3. Network support – L2 stretch, IP mobility, cost!
4. Split brain – occurs at storage layer and VMware layer (disconnected but both still alive, VMs running on both sides without knowing if each half is still alive)

Stretched Storage Configuration

• Stretch SAN fabric between locations (clustered storage model split between geographically-dispersed sites)
• Requires synchronous replication (<5ms latency, limited to ~100km distance, every I/O incurs latency impact)
• Read/write on one side and read-only on another side (VMs could potentially be reading/writing to storage over network)

Distributed Virtual Storage Configuration

• Distributes read/write storage in both locations with data locality algorithms
• Requires synchronous mirroring
• Multiple controllers in scale-out fashion (must address multiple split-brain scenarios)

Stretched Cluster Considerations

1. VMs can incur storage performance penalty if they access storage at partner site (not applicable with distributed virtual storage)
2. “Sidedness” or affinity to control HA/DRS behavior
3. Operational considerations in maintaining DRS host affinity rules
4. No supported way to control HA primary/secondary with vSphere 4.x (vSphere 5 mitigates this with FDM)
5. Requires layer 2 equivalence at network layer (OTV, VPLS/L2 VPNs)
6. Network lacks site awareness (leads to trombone routing as VMs go from one site to another)

Additional Reading

For a comprehensive and well-articulated breakdown on this topic, see Chad’s article: Understanding vSphere Disaster Recovery/Avoidance Options.

Also, one of my personal favorite reads that extensively explores stretch cluster failure domains was published by NetApp and VMware in the technical report titled: A Continuous-Availability Solution for VMware vSphere and NetApp

Traditional Backup and Recovery

Pros

• Easy, inexpensive – just add vCenter to backup schedule
• Low complexity
• Minimal or no additional licensing (not really the case for third-party solution, vDR included in all versions of vSphere except Essentials)
• Supported for vCenter Server Appliance

Cons

• Only a DR solution, not an availability solution
• Insufficient for planned downtime
• Manual recovery
• Complexity (SSL certificates, database protection)

Cold Standby Server

Pros

• Easy if vCenter is a VM and DB is local
• Recovery time shorter than backup/restore solution
• Replicate to another local host or remote site
• vCenter Appliance can be protected

Cons

• Application consistency could be an issue
• Manual recovery
• Garbage in/garbage out (corrupt data on hot server replicated to cold standby)
• DR plan required

Clustering Solutions

Pros

• Provides H/W, OS, and partial application protection
• Flexibility between virtual and physical servers

Cons

• Support – MSCS (Microsoft Cluster Services) and VCS (Veritas Cluster Services) not certified by VMware, best effort support
• Complexity
• Limitations (no NFS, FCoE, VMware FT)
• Additional licensing

VMware HA and Application-aware API Solutions

Symantec ApplicationHA
Neverfail vAppHA

Pros

• Provides hardware (HA), OS (blue-screen), and application-level protection (service failure)
• Easy to deploy, configure, manage
• Automated recovery
• Extends capabilities of HA
• No requirement for second node

Cons

• Licensing costs
• Support from multiple sources
• Does not protect against network failures or performance issues
• Not applicable for vCenter server on physical server
• OS patching still requires downtime
• DR plan still required

VMware vCenter Heartbeat

• Purely software-based solution
• Shared-nothing architecture
• Active/passive nodes
• vCenter application awareness
• Physical and virtual support
• Hardware and software redundancy

Pros

• Robust protection against hardware, OS, network, and application failures, as well as application performance degradation
• Awareness of all vCenter components (including VUM, Composer, Converter, Orchestrator and database tiers)
• Only solution fully supported by VMware
• Deploy in LAN (both nodes in same DC) or WAN architecture (secondary node at remote site)
• Protects against planned (switchover) and unplanned downtime (failover)

Cons

• Additional licensing cost
• DR plan needed – not a replacement for backup and recovery

My personal editorial comments:

1. Backup/restore is not the same as disaster recovery – your backups are a point in time copy that can be restored at will, your disaster recovery plan is the set of procedures and steps (run book) to mitigate a complete infrastructure failure
2. Availability is not the same as disaster recovery – if you were to think about it in terms of the vSphere stack protecting virtual machines, availability is HA, disaster recovery is SRM.  If you were to apply the same concept to vCenter, availability is a cold standby, cluster solution, HA w/ or w/o application HA, or vCSHB.  Disaster recovery is SRM or restoring from a replicated backup at a recovery site.
3. If budgets are constrained and RTO/RPO is within a tolerable level (minutes as opposed to seconds), organizations should consider HA (with the option of application-aware APIs) and a long-term disaster recovery solution.  This could simply mean HA providing local site availability and SRM providing a disaster recovery solution.
4. Large-scale enterprises that depend on vCenter as a mission-critical application to manage virtual infrastructure components should leverage a continuously-available and enterprise-proven (and supported) solution.  That solution is vCSHB protecting both the application and database tiers of vCenter.  An important consideration to note: for active/active datacenters with minimal latency (think stretched VLANs), vCSHB deployed in LAN mode with a single node at each active site could provide both high availability and disaster recovery.  In this design, either a single node failure or a complete site failure would both result in a failover to the secondary node.

• SRM is designed to be the orchestrator, the tool that helps plan for recovery and provides one location to manage and test recoveries
• Testing guarantees that environments will run correctly when disaster hits

Technology – vSphere Replication (VR)

• Previous SRM versions were dependent on SRAs to direct replication at storage layer – still premiere means of protecting environment, far more options and a more mature technology, downside is cost (specifically for SMB market)
• vSphere replication is a component of the vSphere platform itself, embedded in the ESX 5.0 hypervisor and called by SRM
• Underlying storage is irrelevant as VR operates at SCSI-filter level at hypervisor level – VM writes to VMDK disk, filter intercepts, tracks changes, and tells ESXi server which blocks need to be replicated
• vSphere replication will work with any type of disks and heterogenous datastores (i.e. SAN to local disk)
• vSphere replication is a property of a VM itself, set as a policy for a VM instead of managing storage layer, completely transparent to VM, negligible CPU impact to track changes in very busy VM
• Array-based replication and vSphere replication are independent from one another and can be used in parallel
• Granularity: can protect entire VM or select certain VMDKs

vSphere Replication Recommendations

• Don’t use VR replication for VMs sitting on array-based replication, waste of space and complicates protection groups (if LUN is failed over, then VR is broken)
• Seed the environment to minimize initial synch, saves bandwidth and time
• Doesn’t fit every single need, fits RPO between 15 minutes and 24 hours = maximum RPO is 15 minutes (array-based replication is 0 minutes with synchronous replication)
• Snapshots are supported but are not represented at the recovery site, they are collapsed (as a general best practice, limit snapshots whenever possible)

vSphere Replication Limitations

• VR based on SCSI-filter technology with agent and filter on ESX 5.0 hypervisor = only applicable for powered on virtual machines
• ISOs/floppy images not replicated
• Powered-off/suspended VMs not replicated
• Non-critical files not replicated (logs, stats, swap, dumps)
• FT, linked clones, templates, and physical RDMs not supported
• Reprotect and failback only compatible with array-based replication (this is a function of the SRA)

SRM Architecture with vSphere Replication

• Framework: VRS and VRMS virtual appliances deployed and configured through SRM
• vSphere Replication Management Server (vRMS) – controls and coordinates replication, tracks all replication policies, does not partake in replication, just manages it (own database)
• vSphere Replication Agent (vRA) – tracks blocks that change, keeps bitmap of dirty blocks, groups change blocks together and ships across wire
• vRA doesn’t cache blocks but tracks changes and replicates on the fly according to algorithms
• When blocks are replicated, they go straight from ESXi server (vRA) to vSphere Replication Server (vRS) at recovery site, can scale to more than one VRS to distribute writes to ESXi servers to write out to VMDK
• Bandwidth requirements difficult to estimate, dependent on churn rate and RPO (white paper to come on industry-standards)

Scalability

• Maximum of 500 VMs with vSphere Replication, 1000 total
• 10 simultaneous running recovery plans = more VMs recovering in parallel
• SRM 5.0 changed start sequence, previously can start 2 VMs at the same time per host up to 10 hosts = 20 at once, SRM5 changes this as vCenter handles start sequencing if clusters can handle massive power-ons
• Potential problem: delays due to parallel activities can have adverse chain reactions
• Maximums are recommended, but not enforced

Workflows

• Planned migration was #1 use case for SRM – taking set of applications and moving them from one location to another in a controlled fashion
• DR event = aggressive RTO (get VMs up and running as fast as possible)
• Planned migration = aggressive RPO (data-consistent migration, not a failover), guarantees synchronization
• Reprotect is function of SRAs (22 functions in total), reverses direction of replication
• Failover process is a sequence of 4 events: failover, reprotect, failback, reprotect
• Potential problem that depends on how long you’ve been operating at Recovery Site: new VMs introduced, storage vMotion operations, etc

UI Changes

• Unified SPOG in one location for both Protected and Recovery sites (Linked Mode not required)
• Shadow VMs identified with lightning icon
• IP addressing is now managed as a property of the VM at the Protected Site and Recovery Site (remember to set for both sites to cover failover and failback IP addressing), use GUI for one-offs and simple changes, bulk import of IP information better for mass sets of VMs

Miscellaneous

• IPv6 supported, including vSphere Replication
• IP customization performance increase, previous versions used sysprep and customization specifications
• SRM 5.0 now uses VIX API call to inject IP addressing directly into virtual machines providing dramatically improved RTOs
• IP addressing procedure with VIX API: isolate virtual machine, preboot it, inject API call, and shut it down
• VIX API dependent on VMware Tools
• Can now call scripts within a virtual machine for application specific customizations
• Five priority groups – VMs within priority groups start in parallel
• Dependencies within priority groups create matrix for start sequencing, does away with complex and numerous recovery plans, use sparingly as dependencies compromise the benefits of parallel start sequencing
• vSphere Infrastructure Navigator uses VMware Tools to sniff application interdependencies between virtual machines

And the atomic bomb dropped at the end of the session as most people already left for their next session…

• Storage vMotion and Storage DRS are not supported with SRM 5.0.  Period.  Issue: VM could migrate to a location that is not protected, therefore compromising recoverability.  More on this from Cormac Hogan.

iSCSI/FC (block) storage – SRM initiaties a rescan on the HBAs on the hosts used for recovery.  This action is performed in parallel, meaning the HBAs of all recovery hosts will be rescanned simultaneously and then presented the replicated LUNs immediately.

NFS (file) storage – SRM mounts the replicated NFS volumes on the ESX hosts during the recovery.  This action is performed in a serial manner, meaning each ESX host must individually mount all replicated datastores one by one.

Example:

• Assumption: It takes 7 seconds to mount a single replicated NFS volume on a single ESX host at the recovery site
• There are 20 replicated volumes to mount at the recovery site
• There are 10 ESX hosts in the recovery site cluster

The amount of time to mount all replicated volumes across all hosts during a recovery = 7 seconds * 20 volumes * 10 hosts = 1400 seconds (~23 minutes)

~23 minutes to mount all NFS volumes as opposed to ~3-5 minutes to rescan all host HBAs for iSCSI/FC LUNs.  For large-scale environments with a substantial amount of hosts and NFS volumes, this could have a significant impact on your recovery’s bottom line: application SLAs and infrastructure RTOs.

As such, when designing SRM solutions with aggressive timelines:

1. Stick to fewer, larger NFS volumes to decrease the duration of time taken to mount multiple recovery volumes
2. Consider having fewer protection groups (remember, protection groups are based upon replicated datastores) to reduce overall recovery time

For further design impacts when engineering your disaster recovery plan, see the VMware vCenter Site Recovery Manager 4.0 Performance and Best Practices for Performance white paper.

Add vMA to a domain

1) Log in to vMA console as vi-admin
2) Add vMA to the domain

sudo domainjoin-cli join <domain> <domain admin user>

3) When prompted, provide the Active Directory administrator’s password (for the user entered in Step 2)
4) Restart vMA
5) Login to vMA console as vi-admin and confirm the domain join was successful

sudo domainjoin-cli query

You should see the following returned:

Name = <vMA name>
Domain = <domain name>
Distinguished Name = CN=<vMA name>,CN=Computers,DC=<domain name>,DC=<top-level domain name>

You should now be able to login to vMA with Active Directory credentials.

Add a vCenter Server system as a vMA target for Active Directory Authentication

1) Log in to vMA console as vi-admin
2) Add a vCenter Server as a vMA target

vifp addserver <vCenter server> --authpolicy adauth --username <domain>\\<domain user>

Note the second backslash (\\) to escape the first!

3) Verify the target server has been added

vifp listservers --long

You should see the following returned:

vc.mycompany.com   vCenter   adauth

4) Set the target vCenter as the default for the current session

vifptarget --set vc.mycompany.com

You should now be able to execute a vSphere CLI command against any of the ESXi hosts managed by the vCenter target without needing to authenticate to an individual host.

esxcli --vihost <esxHost1> network nic list
esxcli --vihost <esxHost2> storage nmp device list

Further information can be found in the vSphere Management Assistant Guide.

“Failed to create database tables. Details: DB does not contain SRM schema: VdbError: Column name does not exist in table dr_product_info”

Microsoft SQL Server Configuration

Microsoft SQL Server configuration must meet specific requirements to support SRM.

1. SRM requires that the Microsoft SQL Server must have a 32-bit DSN.

• %WINDIR%\syswow64\odbcad32.exe

2. The database schema has three requirements:

• It must be owned by the SRM database user (the database user name you supply when configuring the SRM database connection).
• It must be the default schema for the SRM database user.
• The DB schema name must be the same as the DB user name.

3. You must grant the SRM database user the following permissions:

• bulk insert
• connect
• create table

4. If you are using Windows authentication, the SRM server and database server must run on the same host.

5. If the SRM server and database server run on different hosts, you must use mixed mode authentication.

6. If SQL Server is installed locally, you might need to disable the Shared Memory network setting on the database server.

To verify the above database schema requirements:

1. Login to Microsoft SQL Server Management Studio
2. Select Databases -> <Database name> -> Security -> Users -> Right click <Database user> -> Properties
3. Ensure the following:

• Default schema is that of the database owner username (i.e. svc-srm01 database user’s default schema is svc-srm01)
• Schema owned by this user is the schema named that of the database user (i.e. svc-srm01 database user owns schema svc-srm01)

Additional information regarding SRM database configuration requirements (including Microsoft SQL Server, Oracle Server, and IBM DB2) can be found in the Site Recovery Manager Administration Guide for vCenter Site Recovery Manager 5.0 (page 28).

Supported number of adapters for vMotion in vSphere 5 (per host):
1GbE – 16 NICs
10GbE – 4 NICs

Concurrent vMotions in vSphere 5 (per host):
1GbE – 4 vMotion operations
10GbE – 8 vMotion operations

The setup:

1) Create a vSphere Distributed Switch with n amount of Port Groups, where n is the number of uplinks being used for vMotion.

In this configuration, I’m using (2) two 1GbE uplinks and a dedicated vDS for vMotion, therefore my Port Group names are vMotion_1 and vMotion_2.  Remember, this scales to (16) sixteen 1GbE uplinks and (4) four 10GbE uplinks, so your naming convention could be vMotion_x, vMotion_(x+1), … vMotion_(n), where n is the number of uplinks being utilized for vMotion.  For environments using multiple 10GbE adapters and a shared vDS for other traffic (i.e. Management, IP storage, FT, vSphere Replication, virtual machine), ensure Network I/O Control is configured to mitigate uplink saturation as a result of vMotion activities.

2) Add your hosts to the vSphere Distributed Switch (either during or after the creation of the switch) and select the physical uplinks to be used.

3) Edit each Port Group and configure Failover Order to use a single Active Uplink.  All other uplinks should be configured as Standby Uplinks.

Example:

• Port Group vMotion_01 uses dvUplink1 as the Active Uplink and dvUplink2 as the Standby Uplink
• Port Group vMotion_02 uses dvUplink2 as the Active Uplink and dvUplink1 as the Standby Uplink

4) Configure p amount of VMkernel ports on each host where p is the number of Port Groups configured for vMotion (this should be the same as the amount of uplinks being used for vMotion in Step 1).  Bind each VMkernel port to a respective Port Group.

Your vSphere Distributed Switch should look similar to this:

Notice that vmk1 and vmk2 are using a single Active Uplink adapter per Port Group.

The results:

(1) One 1GbE network adapter for a single vMotion operation

• 884Mb/s transferred through vmk1
• 26s average virtual machine migration time

(2) Two 1GbE network adapters for a single vMotion operation

• 1774Mb/s transferred through vmk1 (886.74Mb/s) and vmk2 (886.84Mb/s)
• 16s average virtual machine migration time

That’s a 101% increase in raw data transfer and a 63% improvement in average migration time using only (2) two 1GbE network adapters for vMotion.  Although not likely to scale linearly, I presume further gains could be realized utilizing additional 1GbE adapters, or even more so with 10GbE adapters.

The conclusion:

Multi-NIC support for vMotion will dramatically improve virtual machine migration times, thus allowing for:

1. increased resource efficiency and allocation within a Fully Automated DRS cluster
2. improved response times for evacuating hosts for maintenance (specifically significant in large VDI environments where increased densities are more prevalent)