30 April 2013

Creating boot USB stick with Windows 7 download tool fails on brand new USB drive

Loads of new USB flash drives come "prepared" with all manner of stuff on it.  If like me, you often just want to use the USB drive as OS build device you need to have it in a "native state" before you can use the Windows 7 USB/DVD download tool

http://images2.store.microsoft.com/prod/clustera/framework/w7udt/1.0/en-us/Windows7-USB-DVD-tool.exe

You will know that your USB drives needs to be when you receive the following error message

"We were unable to copy your files. Please check your USB device and the selected ISO file and try again."



DiskPart
You can resolve the issue using diskpart with the following list of commands

diskpart
list disk
select disk 1
clean
create partition primary
select partition 1
active
format quick fs=fat32
assign
exit

This assumed that the USB drive was identified as disk 1

PowerShell
Alternatively you can also use PowerShell in Windows 8 or Server 2012


Get-Disk 
Initialize-Disk -Number 1 -PartitionStyle MBR
Clear-Disk -Number 1 -RemoveData -RemoveOEM
New-Partition -DiskNumber 1 -UseMaximumSize -IsActive -MbrType FAT32 
Get-Partition -DiskNumber 1
Get-Volume
Format-Volume -DriveLetter f -FileSystem FAT32


This assumed that the USB drive was identified as disk 1 and that the partition was assigned drive letter f

24 April 2013

Windows HotFix management with PowerShell

Checking if a particular hotfix / update / patch  is installed is often an important post-build check.  With non GUI Windows server deployments you can use the following command lines to get the info you want.

Install from the command line
You can individually download and install windows updates by KB number.  Simply go to download.microsoft.com and search for the KB number.  Once downloaded you can use the Windows Update Standaline Installer to install the update.

wusa .\Windows8-RT-KB2822241-x64.msu /quiet


Retrieving list of installed updates
To get a full list of installed hotfixes run

get-hotfix




Get info on specific KB

Get-HotFix -id KB2803676

If you are looking for a specific few updates you can use a variable

$numbers = "KB2770917","KB2779768"
Get-HotFix -id $numbers

If you want to check for updates installed after a specified date

Get-HotFix | ? installedon -gt 2013/04/01

*Thank to Windows Scripting guy

Alternative commands

There are of course a few other toold that can give you a list of hotfixes but they
You can query the WMI with wmic

wmic qfe list


Our you can run good old systeminfo

systeminfo




22 April 2013

Determine Windows system up-time

Knowing how long a Windows server or even desktop has been up for can be very useful.  These command lines will help you find the relevant info you are after.

   net stats srv | find "since"

or

   systeminfo | find "System Boot Time"



Both will return the system boot or running time and from there you can just work out the up-time.  Easiest is just to put the boot date and the current date into Excel subtract the one form the other and it will give you the amount of days.






18 April 2013

Manipulating vhd and vhdx with powershell

Everything you want to know about how to create,convert, re-size, mount and attach Hyper-V disks.

The Hyper-V management gui provides great simple to use tools for basic virtual disk manipulation, if however you want to get serious you break out the powershell cmdlets.

Creating a virtual disk
To start things off I will creat a new dynamic disk that is 25GB in size.

New-VHD -Path 'b:\test.vhdx' -Dynamic -SizeBytes 25GB 

To check the properties of the disk as we progress we will keep referring to

Get-VHD .\test.vhdx

After the initial creation we will see


ComputerName            : ET-LAB-HV03
Path                    : B:\test.vhdx
VhdFormat               : VHDX
VhdType                 : Dynamic
FileSize                : 4194304
Size                    : 26843545600
MinimumSize             : 
LogicalSectorSize       : 512
PhysicalSectorSize      : 4096
BlockSize               : 33554432
ParentPath              : 
FragmentationPercentage : 0
Alignment               : 1
Attached                : False
DiskNumber              : 
IsDeleted               : False
Number                  : 

Converting between vhd and vhdx
The only  thing you need to specify is the extention, the correct type is used.

Convert-VHD .\test.vhdx -DestinationPath .\testFixed.vhd

Converting between dynamic and Fixed
This is controlled by the -VHDType parameter

Convert-VHD .\test.vhdx -VHDType Fixed -DestinationPath .\testFixed.vhdx

Attaching a VHD to a virtual machine

Add-VMHardDiskDrive -VMName TestVM -Path B:\test.vhdx

Mounting a virtual disk to a physical host
Server 2012 introduced to feature of being able to mount a vhd as a disk form the OS.

mount-vhd .\test.vhdx 

The disk will now appear as a disk to the operating system.   Since created the disk but no partitions there is no associated volume.

Initialize the disk

Initialize-Disk -Number 1

Create the partition

New-Partition -DiskNumber 1 -DriveLetter j -Size 10GB

Create the volume

Format-Volume -DriveLetter j -FileSystem ReFS

Now that we have a volume on the disk we have a new attribute.  if your run get-vhd now you will see a value for  minimumsize

Un-mount the virtual disk

Dismount-VHD -DiskNumber 1

Re-size the fixed disk to a certain size
The vhd can only be as small as the minimumsize (Volume size) but it can be bigger

To shrink or grow the file to a predefined size use

Resize-VHD .\testfixed.vhdx -SizeBytes 15GB

Or to make it is small as possible use

Resize-VHD .\testfixed.vhdx -ToMinimumSize

Conclusion
Using powershell allows you to create and manage virtual disks at all it's life stages.  It also provides more functionality that you have using the GUI tools and allows you to break away from predefined default size restrictions.

15 April 2013

F5 Diaries - Episode 4 - Really Simple Inbound Load Balancing with GTM

Outbound load balancing is simply a case of putting a few internet lines together and sending traffic out and
let it route back they same way.  Inbound load balancing is a bit harder.

How it works
I am using the same lab form Episodes 1,2 and 3.  From this we know that we have two Internet connections.  If we want to be able to publish a web site on either connection it would need an IP address associated with each link.

The same site would therefore be available on two different IP addresses.  This is where GTM comes in.  It selects the IP address to return to the requester and this governs over which connection the conversation will happen.



Configure the F5
There are of course many ways of doing this, this guide will cover the basics and this should give you enough understanding to carry on building more complex deployments.
  • Provisions the GTM module
  • Create a datacentre
  • Create links
  • Create listeners
  • Create a server
  • Create a virtual server (LTM)
  • Create a GTM pool
  • Create a Wide IP
At the end of this you would have a DNS server on the Internet that would return an IP for the site  (virtual server) you created.

Provision the GTM module
The F5 BIG-IP Ve Lab edition on TMOS 11.3 is licensed for GTM, but it is not provisioned by default.
  • System
  • Resource Provisioning
  • Check Global Traffic Management and select Nominal fromt he drap down
  • Submit
  • Reboot
Create a Data Center
Data centres act as coitainers for the various object and allows them to be logically grouped.
  • Global traffic
  • Data Centers
  • Data Center List
  • Create
  • Specify a name such as Data_Centre1
  • Finish
Create Links
Links represent physical connection links to the internet.  In the lab we have two ADSL lines, one with a fixed public routable IP the other with a dynamic IP, because of this one link would need translation and the other does not.

Create a fixed public rout-able IP link
Use this for creating a link for a internet connection that will always have the same fixed rout-able IP address
  • Global traffic
  • Links
  • Link list
  • Create
  • Name IS_FixedIP
  • Address Translation Disabled
  • Router address 169.212.74.65
  • UplinkAddress 169.212.74.66 (the ip you get from whatismyip.org)
  • Data Centre Data_Centre1
  • Health monitors bigip_link

Create a dynamic IP link
Note: the Uplink address is the IP that would be visible on the internet for the F5 itself.  If you are not sure go to whatismyip.org and verify.

  • Global traffic
  • Links
  • Link list
  • Create
  • Name Telkom_dynamic_IP
  • Address Translation Disabled
  • Router address 192.168.0.1
  • UplinkAddress 105.221.130.246 (the ip you get from whatismyip.org)
  • Data Centre Data_Centre1
  • Health monitors bigip_link
Create Listeners
Listeners sit on the internet IPs of the F5 and listens for DNS requests.  Listeners will show up a virtual machines in LTM
  • Global Traffic
  • Listeners
  • Create
  • Destination is the self IP
  • Finish
  • Repeat for the second public IP
You should end up with one on 192.168.0.254 and another on 169.212.74.66
These are the IP's and listeners we will be testing later on so keep this in mind when we get to the test pc.

Create a server
A server is a physical device that contains the virtual servers that are ultimately the destination of the DNS request.
  • Global traffic
  • Servers
  • Create
  • Name: GTM_and_LTM
  • Product: BIG-IP System (single)
  • Add both Internet Self IPs to the address list (192.168.0.254 and 169.212.74.66)
  • Data Centre Data_Centre1
  • Health monitor bigip
  • Virtual Server Discovery Enabled
  • Link Discovery enabled
Create a virtual server (LTM)
The server we just created needs contain virtual servers.  The easiest place for this is to create a virtual server on the local LTM.  In the lab I have a IIS web site an internal network.
  • Local Traffic
  • Pools
  • Create
  • Name pool_test_iis
  • Health monitor http
  • New Node (ip address of IIS site)
  • HTTP
  • Finish

  • Local Traffic
  • Virtual Servers
  • Create
  • Name vs_test_iis
  • Destination Host (This ip is what will be returned in a DNS request - so this is a public ip)
  • Service port HTTP
  • Default Pool pool_test_iis
  • Finish
Create GTM pool
A pool is a collection of  virtual server that can reside on multiple servers.
  • Global Traffic
  • Pools
  • Create
  • Name gtm_pool_test_iis
  • health monitor http
  • Memberlist (Select the virtual server created in the previous step)
  • Add
  • Finished
Create a Wide IP
A wide IP is a FQDN "DNS entry" that is mapped to the pool members   This is essentially the dynamic DNS name that can return different ip addresses.
  • Global Traffic
  • Wide IPs
  • Create
  • Name: www.mytestdomain.com (this is the actual fqdn)
  • Add the gtm_pool_test_iis to the pool list
  • Finished
By now you should have a online wide ip.  If you don't you need to go back and check the monitors to make sure that the wide ip is online.

Testing GTM from the Internet
Generally the GTM would be an authorative DNS server on the internet.  To test however we can simulate this by specifying the listener IP in an nslookup.

On your test internet connected PC
Open command prompt
nslookup
server 169.212.74.66  (your public ip specified in the listener)
now search for www.google.com - you should got no answer
now search for yourwide IP FQDN

You should get a reply with the IP specified in the LTM virtual server.

To test the same over the second ADSL link do the same but specify the nslookup server to be the other dynamic IP listed in the listeners.  A note here.  Since the dynamic IP can change it is not recommended to use one.  It may be useful for testing but it is not feasable for any real deployment


11 April 2013

F5 Diaries - Episode 3 ISP Load Balancing

Having redundancy or fail-over is one thing, having active load balancing is another.  F5 LTM allows you to
configure both.

In Episode 1 and 2 I covered building the lab and setting up a very basic ISP redundancy.  In this episode  I will place that configuration in a real world deployment, send some traffic trough it and monitor and tweak setting until we have load balancing.

Real world deployment.
Combining your ISP links is most effective if it is done as close to the perimeter as possible.  This means it normally sits outside the corporate firewall.  Clients would then access the load balanced gateway through a NAT or proxy.


Expected conversations
From the proxy perspective the traffic source will be the individual internal IP addresses of the client computers.

From an F5 perspective all the TCP/IP traffic will have the same source IP.  The one of the Firewall or Proxy.  The destination will be the Internet site.  The F5 will elect the external interface to use to send the traffic to that site.

From the internet site the source IP would be the public IP associated with the link that was used.  If the route was via ISP1 the source would be different if the route was via ISP2

Configuring persistence
Once a TCP conversation is started on one route you would like to maintain that route for the duration of that conversation.  This is persistence.  Normally persistence is based on the client address or session cookie.  However in this case the source IP will always be the same. This is great for persistence but would not allow for effective load balancing.  We can use the destination address since this will be very different from conversation to conversation.


  • Local Traffic
  • Virtual Servers
  • Default_gateway (your wildcard VS)
  • Resources
  • Set Default Persistence Profile to dest_addr


Deciding how to split the load
The ideal is to be able to connect many different ISPs with potentially different performance capabilities.  As an example the lab has 1 x 4MB ADSL and 1 x 512KB ADSL.  Because these links will have varying different performance and load on them you want to set the load balancing method to be a dynamic on that evaluates the connection count as well as the performance.

The Observed (node) method uses a combination of the logic evaluating fastest response time and least connections.


  • Local Traffic
  • Pools
  • default_gateway_pool
  • Members
  • Set the Load Balancing Method to Observed (node)


Test your deployment
This setup work nicely for the lab but your environment will probably different in some regard.  It is important to check and see that your load is being distributed as you would like it to be.

You will need:
A number of client computers attempting to access the Internet through your gateway or proxy.  The wider the scope of sites being requested the better.

One of the easiest places to check that load is going to the various ISPs is the monitor the pool


  • Statistics
  • Module Statistics
  • Local traffic
  • Statistics Type Pools
  • Enable Auto Refresh


After letting traffic run for a bit this is what I could see.  It is as expected with the Telkom line has theoretically 8 x bandwidth of the IS line



Monitor traffic using tcpdump
Seeing traffic being split across the  nodes is useful.  It gives you a high level overview to see that things are going as expected.  If however you want to get a closer look as to what is actually goimg where you will need to check out TCP flows with tcpdump in an SSH session

To see outbound traffic from both of the internet facing self ips you will need to specify the following

tcpdump  src host internetIP1 or internetIP2 -i any


T narrow down to HTTP and HTTPS traffic only

tcpdump src host internetIP1 or internetIP2 and dst port 80 or 443 -i any


To see traffic outbound and  inbound for HTTP and HTTPS

tcpdump host internetIP1 or internetIP2 and dst port 80 or 443 -i any


For a bit more info on tcpdump check

http://support.f5.com/kb/en-us/solutions/public/0000/400/sol411.html


10 April 2013

F5 Diaries - Episode 2 ISP Redundancy

Having more than one ISP for your corporate is essential. If one ISP fails for whatever reason the show must go on.  Similarly if one of the ISPs becomes congested and start to run slowly traffic needs to automatically be routed to the other faster ISP.

This is called outbound load balancing since the connection is initiated form the internal and is requesting an external public resource.  For this functionality you only need F5 LTM you do not need GTM or Link Controller

This lab is going to build on the standard lab from Episode 1 http://fixmyitsystem.com/2013/04/f5-diaries-episode-1-building-lab.html

As a guide I am going to use this F5 doc http://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm-implementations-11-3-0/2.html#unique_2145903433

How it works
To be able to test this  you will need to a machine that uses an F5 external Virtual server as it's default gateway.  All request are received by the F5 and then load balanced to the external ISPs.  The response is then received by the F5 and routed back to the original client source.



Configure the F5
Summary of what need to be done

  • Create ISP nodes
  • Create an ISP pool
  • Create a virtual server to act as default gateway
  • Enable SNAT automap


Create ISP nodes
You need to create a node for each ISP's ROUTER address

  • Local Traffic
  • Nodes
  • Create
  • ISP1 (Name:  Telkom; Address: 192.168.0.1)
  • ISP2 (Name:  IS; Address: 169.212.74.65)



Create ISP Pool
Use the nodes to create a pool for the virtual server

  • Local traffic
  • Pools
  • Create and name it default_gateway_pool
  • Add the gateway_icmp as the health monitor
  • Load balancing method Round Robin
  • Add the two ISP nodes for * All services
  • Finish


Create Virtual Server
You need to create a wildcard virtual server to act as the default gateway for all connection outbound to the Internet.

  • Local traffic
  • Virtual Servers
  • Create with the following
  • Name: vs_default_gateway
  • Type: Performance (Layer 4)
  • Source: 0.0.0.0/0
  • Destination: Network
  • Address 0.0.0.0
  • Mask 0.0.0.0
  • Service * All Ports
  • VLAN and Tunnel Traffic : Enabled on Internal VLAN
  • Source Address Translation: Auto Map
  • Default Pool: default_gateway_pool


Enable SNAT Auto Map
This allows traffic to rounte correct between the VLANS

  • Local traffic
  • Address translation
  • SNAT list
  • Creat and call it SNAT
  • Translation Automap
  • VLAN Enabled on External & Internal



Configure the Test PC
The test PC now needs to use the F5 as it default gatway to the Internet

  • Connect the Test PC's NIC the the Internal virtual Switch
  • Configure the Network with a static IP details:
  • IP  10.0.7.252
  • MASK 255.255.254.0
  • Gateway 10.0.7.9 (This is the self IP of the F5's Internal VLAN)
  • DNS 8.8.8.8 (Google Public)


Testing
At this point everything should be working.  One the test PC open a browser and connect to http://www.whatismyip.com.  This should give you a public IP.

To check that you have failover disconnect one of the physical DSL routers.  You should now see the pool members being flagged as down, but your internet browsing should still be working.  I did notice a little delay as things were cutting over.  Do the reverse now with the other DSL router. You should observe that the public IP has changed

Troubleshooting
If your browser is not returning any pages check the following

Problem: F5 Returns traffic but no DNS, sites work on IP but not name

  • Open a command prompt and do an nslookup www.google.com
  • You should get a list of IP's if you don't get the list of IP's form another machine.
  • Use the IP in the browser instead of www.google.com
  • If the page loads you know that DNS look up is broken
  • This happens because the SNAT auto map for the Internal And External VLANS has not been defined.
  • Configure the SNAT as described above





09 April 2013

F5 Diaries - Episode 1 Building the lab

Join me on my journey of trying to make F5 kit work without going on all the official (read - expensive) training.  I am using a test lab environment specifically set up for doing all this kind of testing.

What is in the LAB

  • F5 BIG-IP Ve Lab edition on TMOS 11.3
  • The virtual edition is running on Windows Server 2012 Hyper-V
  • Windows 8 test VM


Hyper-V VM Configuration :

  • RAM 4GB Static
  • CPU 4 x Virtual
  • HDD 1 x 40GB
  • NIC  7 Virtual assigned  5 x Physical


Network Environment:

  • Internal Network is on the 10.0.0.0\8 address range
  • Internet ADSL Link 1 from Telkom is on 192.168.0.0\24
    (GW 192.168.0.1)
  • Internet ADSL Link 2 from Internet Solution is on 169.212.74.65\29
    (public route-able )
    (GW 169.212.74.65 )


    Internal network segmented in VLANs

  • 10.0.2.0\23 Corporate Management
  • 10.0.4.0\23 Corporate External
  • 10.0.6.0\23 Corporate Internal


Hyper-V virtual switches
I have configured individual virtual switches for each VLAN. I then managed to map those to BIG-IP interfaces.  This took a long time but it is really important to get this 100% correct.


  • NIC 0  - Corporate Management - Management
  • NIC 1 - Corporate Internal - 1.1
  • NIC 2 - Corporate External - 1.2
  • NIC 3 - Telkom ADSL -1.3
  • NIC 4 - Internet Solutions ADSL 1.4 
  • NIC 5 - Not Connected
  • NIC 6 - Not Connected


F5 BIG-IP Self IPs and VLANs
  • Management IP 10.0.2.9
  • Corporate Internal - 10.0.7.9
  • Corporate External - 10.0.4.9
  • Telkom - 192.168.0.254
  • Internet solutions 169.212.74.66
Windows 8 Test VM
Since there will be many different scenarios in which you would like to test different connectivity options I have also included a virtual test PC on the same Hyper-V server. Depending on the test or lab configuration this machine's single NIC can be connect to any of the Hyper-V virtual switches. As a default it is connected to one of the corporate VLANs

Current Diagram
This is what the lab looks like at this point



Saving the basic configuration
At this point you should be able to connect to the management IP and you should be able to ping the various gateway IPs from an SSH session.

This will be the BASE configuration from where all the different lab build will happen. Take a backup or snapshot and keep this as the starting point for all future lab testing.  Make sure you are very comfortable with snapshots.  As an example, I very quickly got the lab to this state.





03 April 2013

Advanced IIS compression configuration

Enhancing the standard static and dynamic content compression can improve your site's performance.  The standard "out of the box" configuration works great for most situations but every site is slightly different and as such there is always room for fine tuning.

The Basics

Install the compression components
The enable compression on your IIS site you need to have the IIS performance features installed.  To add these open the Server Manager then:

  • Add roles and Features 
  • Select Role Based 
  • Server Roles 
  • Web Server IIS 
  • Web Server
  • Performance 
  • Static content Compression and Dynamic Content Compression


Configure the default settings
There are a few global settings you can change quite easily.

  • Open the IIS management Console
  • Select The Server level
  • Open Compression


Here you have a few options that pertain to whether to enable static and or dynamic content compression.

You can also specify a minimum file size to compress / compressed cache location and a size limit for the amount of content to keep.

Compressing tiny files often does not yield any benefit so just ignoring those is recommended.
The amount of compressed content to keep is 100MB by default.  If your application generates loads of data you would probably want to increase this.

Advanced Configuration
If you are going to be making changes to these setting you need to know what is happening.  I strongly recommend using HTTP Watch to monitor your progress as I found it the best tool when tweaking these settings.

If you want to change additional setting you will have to open the IIS management console.  Select the server level, open Configuration editor.  From the Section drop down select System.Webserver/httpCompression.


There are 17 setting here that can be changed.  Most of them typically would not need to be changed at all.  The fields I am normally interested in are


  • dynamicTypes
  • staticTypes


dynamicTypes and staticTypes refer to the MIME types that will and won't be compressed.  Something to keep in mind is that IIS will only serve MIME types that are approved.

Adding an additional MIME type for compression
As an example we will add bmp files as a content type to compress


  • Open the staticTypes field
  • Click Add
  • enabled True
  • Mime type image/bmp
  • Close the windows
  • Open the dynamicTypes field
  • Click Add
  • enabled True
  • Mime type image/bmp
  • From the Actions Pane click Apply
  • Restart IIS with iisrest


If we look at a .bmp download now we can see that the content is being compressed

In this example you can see that we saved 93.8 % of the required bandwidth.

Choosing the MIME type are not always straight forward.  Some files are already compressed an example of this is .jpg images. Here you can see how compressing the .jpg actually makes it bigger!

If you would like to compress all other images and exclude .jpg add the following to the dynamicTypes and staticTypes field


  • Enabled = False | MIME Type = image/jpeg
  • Enabled = True | MIME Type = image/*



CPU concerns
All the compressing is great for saving bandwidth but it adds to CPU load.  In my experience the additional load negligible given the massively powerful CPUs available nowadays, You can however set thresholds to guard against CPU congestion due to compression overhead


  • dynamicCompressionDisableCpuUsage
  • dynamicCompressionEnableCpuUsage


  • staticCompressionDisableCpuUsage
  • statricCompressionEnableCpuUsage


These set the upper threshold to disable compression and the resume threshold.  Literally the other side of the story is the client device,  the client would incur a similar decompression overhead.  Since the client is only concearned with it one conversation it is not normally an issue, but it is a good thing to keen in mind.

Conclusion
Compressing HTTP content can be a huge bandwidth saver, if you get to know your web application properly you can tweak out some additional performance gains.  Be cautious though, because being over optimistic can actually take you backwards.

If you would like to know more about the other setting available in the configuration editor check out http://www.iis.net/configreference/system.webserver/httpcompression