Tuesday, May 31, 2011

Red Arrows at Duxford BoB day September 2010

It's a long time ago, but last year Mrs B and I went to Duxford so that I
could relive part of my childhood. As a kid I grew up near what had been RAF North Weald and every year there was (as I recall) a massive Spitfire display. Last year to commemorate the 70th anniversary Duxford was to have 16 flying Spitfires - so I had to go.

During the day the RAF Red Arrows flew their usual masterful display, and these are some of the shots I got...












Scheduled tasks not running in Windows 2003 Server Domain Controller


I’ve a couple of virtualised DC’s in the farm that (to make AD restore much easier) do a daily backup to a second partition which then gets farmed out for protection.
The purpose is that when you start a VM in DRSM mode access to backups is, ahem, somewhat tricky.  So by having a partition within the machine with the latest backup (or a restored copy of an earlier one…) can mean a simple NTBACKUP restore can be swiftly executed to give peace in our time.
Well… for some time one of the machines had tasks in scheduled tasks that “Could not start”, and as an added function when you inspected the task properties you’d be prevented by error 0x8009016.
Now there are oodles of ideas out there, and over the past few months (yes, months) when I found time I would give it a go.
Finally today, it seems to be fixed.
Solution Pointer:
Method:
Taking the hint about stopping services, I went for broke and ran this PowerShell command
# stop all running services
Get-Service * | where {$_.status -eq "Running"} | Stop-Service –Force
Thus the server was reduced to absolute minimum of operating features.
Then taking step 6 on its own
“Delete all of the files in the ‘C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\S-1-5-18’ folder”
But I did (and do recommend) take the more precautious approach of renaming the single file contained therein.
Then a start of all the automatic services by
# start Automatic services that are not Running:
Get-WmiObject Win32_Service | Where-Object { $_.StartMode -eq 'Auto' -and $_.State -ne 'Running' } | start-service
And tried to inspect a task’s properties, and it failed.
On further inspection I found the RSA folder was still empty, so decided a reboot would be worth it.
After the reboot, the RSA folder was populated, the task properties were available again, but the task would not run.  A quick delete and re-create of the task and Robert was my mother’s brother again.
Phew!

Monday, May 30, 2011

Photo test - Iceland back in the 35mm days

Dettifoss - Europe's most powerful waterfall.



The south east coastline - the White dots are gulls!


Vik



The waterfall from the Scottish Widows ad



Thingvellir - the location of the first national parliament site




And all scanned from 35mm negatives

Oh, and the little red dots - Mrs B!

all (c) PJ Bryant. No commercial use without written consent


Preparing for a new world...

To be honest, i feel in the minority using blogspot rather than Wordpress with all it's fancy gizmos. But i started there, and shall continue there for now.

But earlier today i had the obvious (but so far not identified) thought that there must be something I can do that is better than emailing in each new post; and be a bit more Wordpress-like in having posts in production.

So a quick trip to the iPad AppStore, and here I am with BlogPress. Let's see how that goes,

I have a couple of photo blog entries i want to create, so i will have a go here. It has to be better than uploading via the web interface!!!

Wednesday, May 25, 2011

From the recent trip: A few minutes to lights down

Roger Waters, The Wall 2011. Dublin - O2

It's been 21 years since we last saw this show in Berlin. Technology's moved on a bit - from friend's reports of the show I think we'll like it.

More than the alternative on show today - Jedward, Westlife and Obama...

...a few days later
And yes, it was better than i could anticipate. Will put some better piccies up later

From the recent trip: Irish folk at the Oliver St John Gogarty, dodgy photo,

but the best camera is the one to hand...

A dodgy shot, but the best i could get for what was (at last) a more authentic folk sound - 2 fiddles, guitar, accordion and voice. After several renditions around town of Wild Rover and (get this) Oasis' Wonderwall, it's good to hear something good.

I was here a couple of years ago, for a stag weekend. This time i'm a lot more sober :-)

Tuesday, May 24, 2011

From the recent trip: The Doghouse, Howth, Dublin

A cross between Down Under of Whitby Bay, Camden Lock et al.

A day's escape from the city on the day Garret FitzGerald is buried, and the City concludes its preparations for President Obama's dropin.

Good coffee, and a superb bespoke green tea for Mrs B. George Harrison and Led Zeppelin on the music too. A good and promising start :-)

From the recent trip: The best pint in Ireland

And in the best state :-)

Saturday, May 21, 2011

#Definition: Luck

Put bag over head, to place under coat to protect from impending rain storm

In process, rip out ear stud from ear.

Mrs B find clip within seconds.

Rain fails VERY heavily.

Spend 20 minutes searching pavement for (precious in emotional terms) stud [it's a piece of Whitby Jet purchased years ago on a great holiday].
During which the road sweeper goes up and down the pavement whilst you look on in (dripping - it's teeming down) frustration.

Give up

Return to base in anger at losing something precious.

Remove saturated clothing - literally soaked to the skin.

Remove coat and check stud not caught up in folds or pocket. It's not. Shake down. Still not
there.

Take off bag, remove contents and look.

Turn bag upside down and shake out - ear stud falls out, the opening into which it has fallen it about 3/4" square.

Time to buy a lottery ticket!!!

Friday, May 20, 2011

Dad, RIP


We interred Pa yesterday at the Epping Forest Woodland Burial Park.

Sunday, May 15, 2011

Libya and Super-injunctions collide

From the Telegraph today.

In fact the news on the campaign makes this Adams cartoon far closer to the truth.

Sunday, May 08, 2011

Saturday, May 07, 2011

And finally a good view!

"we accept her one of us
we accept her one of us
we accept her we accept her
we accept her one of us"

A slightly better view from the more expensive seats

Well, not really, just got back to a step!

A young person's guide to Marillion

Or the A-Z of Marillion as chosen by the band

Asylum satellite #1
Born to run
Cannibal surf babe
Deserve
Especially true
Fantastic place
(a few snatched phrases of Grendel) Gazpacho
Half the worldv
(a short) Intermission
(a crowd voiced/sourced) Jigsaw
King
Last century for man
Marbles
No such thing
(the) Only unforgivable thing
Pseudo silk kimono
Quartz
(the) Release
Sugar mice
Three minute boy (followed by *T*he *U*naccompanied audience later joined for a jam by the band!!!)
Under the sun
Voice from the past
(joined by Dave Gregory of) XTC Senses working overtime
You're gone
(opening with rock 'n' roll) Zeperated out :-) but there were shouts for Stairway to Grendel!


Actually, I'd like to take a bit of the credit for this (or at least the framework for this). Some months ago when Marillion were planning the weekends Mark Kelly (@markke11y) tweeted asking for ideas. I suggested an alphabetic theme, but based over 2 nights (so that Saturday could end with Neverland). In the end, the band decided to try for all 26 in one night's phenomonal performance (well 3, Port Zelande, Montreal, and Leamington Spa). Kudos for including a track from Fugazi and hoping the crowd would help out (and more importantly remember)!

Having got a rare, near perfect view at the gig...

One gets moved off the steps (alone amongst all the others there - seems I inadvertently *partially* blocked the view of someone important).

To the more usual viewpoint for a not entirely tall person.

^%$$#$

I imagine this was used as an uncomfortable creche for drunks...


And further upstream...

The lifecycle of the butterly in wood

Warwick, just down the river from the castle.

Friday, May 06, 2011

Thursday, May 05, 2011

And finally, some of the other craft in the air that day

During the day, we were treated to many displays. It was a good warm up act!




the noisiest being the modern French fighters


and the Memphis Belle.






Wednesday, May 04, 2011

May i just say, nom nom nom (not to @AndreDang stds though I hazard)

Growing in the garden an hour ago.

Wensleydale, puff pastry

Seasoning

15 minutes.

YUM!!

Monday, May 02, 2011

ESXi 4.1 Update 1 travail - lessons learned.

I’ve been biding my time over the last few months to migrate to ESXi.  Knowing that ESX4.1 is that last edition of the “full fat” VMware, I knew my next move would have to be to ESXi, so rather than make a bigger job whenever (cough) 5.0 is launched. I thought I’d change over the long weekend when I knew clients would be closed.
 
It was entertaining.
 
Building a boot and install USB stick rather than using a DVD burned with an ISO image was an important part of the test.
This is going to come in useful next month as I have some client work then, where the dirty nature of the computer room (a breeze block room in the corner of the warehouse) means that DVD drives become unusable within a few months – I dread to think (and am not responsible for!) the state of the servers and SAN…  So anyway I want to be able to boot and install from USB if necessary.
http://blog.vmpros.nl/2010/09/03/vmware-how-to-create-a-bootable-esxi-usb-stick/ didn’t really work for me, but http://www.ivobeerens.nl/?p=699 proved to be a good source of a procedure on how to do this.  However there are some caveats to the process:
  • Syslinux 4.0.4 (the latest) does not work (or at least did not for me) – stick to 4.0.3!!
  • When modifying the contents of the stick remember to do everything!
  • Whilst the storage in my instance is software iSCSI IT IS IMMENSELY PRUDENT TO DISCONNECT STORAGE.  As this install process initialises some storage, you do not want to accidentally wipe a LUN.  My recommendation is always to build ESX(i) hosts disconnected from storage.  It prevents an easily avoidable mistake.  Likewise I avoid “Boot from SAN” setup.
  • Make sure you follow all the steps. I managed to miss 1 or 2 a few times before I got it right.
  • Don’t forget that the KS.CFG is YOUR INSTALL SCRIPT.  It’s easy to forget this and take the content and run with it.  If you do, you’ll get an ESX box with 192.168.1.10 as its IP, VMware01 as the root password, and ESXi-01.beerens.local as its full name connected to a domain “beerens.local”.  I could be wrong, but I think this is unlikely to work in your world J
 
So once the stick is done:
  • Check for any Anti-Affinity rules in DRS, this will make sure your VM’s can have maximum mobility around the farm during the change.  You may want to weaken them
  • Move any non-running servers off local storage (if there is any) to SAN or other shared storage – cut and paste or storage migrate.  If you storage migrate you can change the host as well to unregister them from the server.
  • Storage migrate all running VM’s on local storage off the server to shared storage (no downtime here).
  • Put the ESX host in maintenance mode (and take the option to migrate all paused and stopped machines off the host).  All running guests will migrate off
This will leave you with a host doing no work, and having no VM’s stored in its local storage.
 
Now, and this is optional, but I highly recommend it.
  • Document the server setup – including network settings, iSCSI paths, vSwitch names and configs.  In fact everything you can!!!  If you are licenced for it, then consider Host Profiles as a means to the end.
  • Disconnect all external storage connections, and verify this by checking via vCentre.
 
 
Now you can start, insert the USB, boot the server, select boot from USB if required and watch it install.  If you have boot from USB as default, then at the end of the install you should remove the USB before it boots again.
Your KS.CFG will do the initial configuration and you have a new ESXi server.
 
This is where some of my fun started.  Now please bear with me – some of this was done late at night over a bank holiday, so I did not do my more normal thorough investigation, and I do not have answers to all the questions, but a list of issues encountered and some observations.
 
  1. vCentre
I thought my vCentre was up to date.  I was lazy, it was not.  I discovered on adding the new host to my network that there were some management issues from VC to ESX.  So I needed to upgrade vCentre.  I also discovered that some VM’s would not start when running on the new host – it seems they were mostly VM Version 4; but also (to make things harder) VMtools needs to be updated too!
 
  1. vCentre upgrade ISO
This is a 2.2GB download.  You do not want to do this on a 512KB ADSL connection.  I hoiked out my 3G MiFi unit, and downloaded it over the air instead to the laptop.  I achieved a 10 fold performance benefit by using this.  Fortunately I had 3.5GB left on the monthly allowance, so all was well.
 
  1. vCentre Upgrade action
Sadly this is a lengthy process, but by using full documentation from the installation (you do have this don’t you?) I was able to breeze through the dialog boxes and get everything up to date except Update Manager.  For some reason that part of the ISO is corrupt.  I am downloading it again as I type.
For prudence I snapshotted the VM that is the VC before starting.  At times later on, I would be tempted to restore to this, put ESX4.1 back on the host and give up.
Oh, and don’t forget to take the in place upgrade option – if you go for a new database your whole farm is screwed! (no, I didn’t)
 
  1. vCentre Client upgrade
On starting the vCentre Client, the new VC edition wants an upgrade before I can connect to it.  This install fails…
Now this was fun… My main management server (physical still – for good historical reasons), is where I do most of the work.   However this is now 6 years old and has a large number of VMware components go through it.  Unfortunately… some old MST file was hanging around and the VI Client upgrade failed.  By now it was late at night after a quick burst of investigation I decided on a more radical approach.  I stopped all VMware services, hacked out all the VMware stuff from the registry, killed VMware folders in Program Files, and rebooted the machine.  This did not completely fix the install, and found a few more VMware folders in the Documents and Settings tree, they went too.
 
  1. DNS and AD failure
Yes, you read that right.  When this box came back DNS was down, and AD was not working as a consequence.  Fearing I’d ripped something out I hadn’t meant to I was tempted to hit the backup tape (you do take backups don’t you?) but waited a bit…
This being more a test lab than a production network the primary physical box on which I was working is the original DC of the network.   The other DC’s are virtual, and it turned out that neither had started properly when I had restarted the ESX hosts a bit earlier.  We had had a power cut earlier in the day, and whilst the kit had all stayed up, it seemed (only with hindsight) that whilst I have UPS’s all round a slight barf on one UPS had impacted a network switch and the virtual world was not talking to the physical world properly.  Taking the IT Crowd “Turn it off and on again” philosophy to its logical limit… I shut down all the VM guests (you do have a PowerShell script for this don’t you?!) and shutdown the hosts.  I then power cycled the switches and waited for them to come back.  I then booted the ESX boxes, and the physical server and all was well.  A quick check round logs and events proved this was the case.
I’m not going to try to work out why, as this was now 1am…
 
  1. vCentre client now installed properly and I can connect to vCentre Server again.
A quick bit of configuration of vSwitches, and all seemed to be well except…
 
  1. iSCSI connections
One of the iSCSI connections relies on decent security from the SAN side – and with the new ESXi installation the IQN’s on the software iSCSI had changed, so the SAN had to be told it was allowed to connect!  A quick fix there, and the new ESXi box can see all storage, and works a treat.
 
  1. Finally all was well
 
  1. So I just need that good ISO for the Update Manager installation so that I can now manage updates across the VM’s (VM Version and VMTools for now).
 
 
Observations?
  • Well you can see from the above that Douglas Adams was right when he wrote “Don’t Panic” – I could have given up with the backups, snapshots and original ESX4.1 that I had and gone back to square one.
  • Document your setup, NOW.  You never know when it might come in useful
  • In ESXi the Service Console no longer exists – look for the Management Network in your ESXi networking setup
  • IQN’s can change
  • Check your VM version – some of your older VM’s may be 4 instead of 7.  In my experience, a VM version 4 had some issues starting and seeing network hardware on a new host.
  • Anti-affinity – keep an eye on it, and restore it when done
  • If you use ESXTOP on ESX, don’t forget – without the service console, you won’t get this on the host
  • ILO – if you have it, make sure you know the password – it saves a lot of hassle connecting to the host
  • Lastly NEVER FORGET you can use the VI Client directly to the host to work things.  If the VC goes down, it means you can still start stop guests, enter/exit maintenance mode, reboot and shutdown an ESX box.  This can be your friend.  A lot.
 
 
Oh, and very lastly – if you finish work at nearly 3am in the morning after some problems like this, then the early morning Radio4 news on the day Osama Bin Laden is killed makes for a pretty good wakeup call.
 

Hurrah - a hibernating Hyper-V laptop!


Well, almost J

I got a new laptop last year and having bumped up the RAM and disk, I wanted to use for a virtualised lab on board whilst travelling or at clients.  Having experimented and asked around on Twitter there was no way (my preferred method) of having Windows 7 with ESXi running under VMware Workstation and then have 64bit guests in vCentre – the VT is not exposed to the ESX guests.  This would have given me the best of ESXi (and a VMware lab), and the VM’s I wanted for carrying a lab in the bag.  VMware workstation was not much use to me as without any memory management I would run out of headroom (although the tree cloned drives would be nice).

A non-trivial additional factor was that I insist on encrypted disks in my laptops.

I then experimented with getting a dual boot world going.  BitLocker and Boot from VHD work well, but not together.  I got a Bitlockered guest machine under Hyper-V as a VHD to boot, but the content was a bit flaky – device drivers).  I then tried getting a dual boot to work with the second boot from a VHD but BitLocker got in the way.  See: Am I really asking too much of Hyper-V  I learned a bit about BDCEDIT along the way.

Eventually after a couple of gotchas/glitches I gave up on the BitLocker VHD or alternate boot option as it was taking too much time (and I had read in a few places I was asking the impossible).    And besides: Word from the wise on BitLocker

Becoming impatient, I then restarted my thinking.  I continued with the Windows 2008 R2 build (Bitlockered drive), with the intent of then building the VM’s that I wanted.

The first bit was to get Windows Server 2008 R2 look more like Windows 7 so it could be my standard desktop-like working world along with some other bits and pieces – I added the following to the machine (some are dependencies):
  • Web Server (IIS)
  • .NET Framework 3.5.1 Features
  • BITS
  • Desktop Experience
  • Ink and Handwriting Services (it’s a tablet)
  • Remote Server Administration Tools
  • Telnet Client (I never usually remember this is off by default!)
  • PowerShell ISE
  • Windows Server Backup Features
  • Wireless LAN Service (it’s a laptop!)
  • BitLocker Drive Encryption
  • Group Policy Management
  • Windows Server Migration Tools (just in case)
I then installed all the usual productivity tools, Office, DropBox, the loathsome iTunes etc. etc.

However, Hyper-V cannot use a Wi-Fi network for external access.  My Lab network is behind a Threat Management Gateway 2010 Server, so only this needs true connectivity.  So a quick bit of research, and I came across the idea of a bridge between the Hyper-V network and the Wi-Fi here: Connecting Hyper-V over WiFi and it works a treat.

So the laptop was where I wanted it to be, the VM’s were being created.  BUT….  You cannot hibernate a Hyper-V machine.  This is clearly a sensible idea, but for the road warrior, it’s more than a nice to have.  To wait for a machine to fully shutdown can be embarrassingly long.

So over to the internet.
The first hit was “Create Dual Boot” solution.  This works by duplicating the boot entry (back to BCDEDIT), and then you choose to run with or without Hyper-V.  Without Hyper-V you can hibernate the machine and bring it back quickly.  But you need to reboot the machine to get Hyper-V back, and then you can start your VM’s.  After that you can run your productivity apps, but can no longer hibernate the machine.  This can be found here: Creating a no hypervisor boot entry on Windows Server 2008

And then I found this:
All you do is the following three steps:
  • Set Hyper-V to start on demand “SC CONFIG HVBOOT START= DEMAND” (note the space after the = sign); then reboot the machine
  • Enable Hibernation “POWERCFG -HIBERNATE ON”
  • Then when you want to run VM’s – “NET START HVBOOT”

Lo and behold.  I have a single boot machine.  Until I start HVBOOT then the machine will hibernate.  Once you have started HVBOOT, then you have to shut down the machine instead, but this is good enough for now.  I’m not certain what impact not running Hyper-V will have on the performance of the machine, but not much I guess.

What next?
Well I guess that I might put VMware Workstation on as well to get some VM’s running whilst still being able to hibernate – maybe just 1 or two so that I can PowerShell in Windows 7 as well…  If only Workstation could use VHD’s (or Hyper-V VMDK’s!!!!)

Oh, and if you try to start a VM without HVBOOT running?  You get this: