Happy New Year.

Its been a nice restful Christmas and I’ve had some time to go through and sort out lots of my photos from the last year. Since I started taking a lot more photographs I’ve been categorising them by year then into sub-folders of when and what we were doing, so for 2014 that was about twenty folders. This barely scratches the surface of the photos I take because most are actually taken with my phone. It goes everywhere with me, so from quick little things that make me laugh, to views when I wish I had a full camera, there’s probably a shot on my phone. All of those images are automatically backed up to Google Plus’ Photo service, which is great as my photographs are on all my devices and it does a nice job of auto correcting some of the limitations of a camera phone. However one thing that is seriously lacking is the ability to get those photographs back out directly from the photo service.

Google offers a TakeOut facility to let you retrieve any of the data Google holds on you. Go to google.com/settings/takeout from there you can de-select everything and pick only photos and press next to create a series of archives. I have two problems with the TakeOut download first it creates a directory for each day of uploads, so potentially that 365 subdirectories for the year. Secondly every uploaded file can be auto-edited potentially several times and each edit is included in the download. Ideally what I wanted was a single directory with just the photographs I’ve taken, PowerShell makes this kind of operation incredibly simple. A quick script later and we are down from 4 Gb to 2.5 Gb and all in one directory.  I’ve uploaded the script to do this to a Gist here.

 

Waverley Abbey

Waverley Abbey in the setting sun

NDC London

A couple of weeks back I got the opportunity to attend NDC London organised by the team between the Norwegian Developers Conference. They’ve branded this event to “new DevelopersConference() London” as Norwegian Developers Conference obviously limits their reach a little bit (to steal a joke from @markrendle‘s stand-up) . I went out to NDC Oslo last year to attend their dedicated security track and was really impressed by the quality of the event and the speakers they lined up so once the registration opened for their London event this year I knew I wanted to attend as much as I could.

Like any conference there are the fantastic talks and the ones where you wonder if you’ve walked into the wrong room. Probably my favourite talk from this years event was Coding Culture by Sven Peters, talking about what makes great engineering team and how to inject enthusiasm and passion into development. His slides and a recording of this talk are here, if you can only watch one this is it.

A few of the other highlights for me were:

  • Jon Skeet demonstrating the scary amount he knows about the Hebrew calendar and confirming date and time is still a pain in the neck. link
  • “Code is like humour, if you have to explain it it’s bad”. A crowd driven refactoring session with Cory House. link
  • “SAML is the Windows XP of online identity”. Dominick Baier talking about OpenID Connect. link
  • Reviewing a year of security howlers and CSI Cyber with Niall Merrigan. link
  • Learning how to count potatoes with drones and how machine learning can optimize farming with Gary Short. link
  • If you are doing exactly what your IDE tells you which of you is writing the code? Along with other pause for thought moments with Kevlin Henney. link
  • “Does your home audio set-up really need an external IP address? Who is adjusting their speakers on the bus?” Looking at the “Fragile Internet of Things” with Erlend Oftedal and Einar Otto Stangvik. link

Videos of all the talks are already available to attendees here and will apparently be made public some time early in the new year.

 

Salisbury Cathedral

Went down to Somerset for a short break last weekend, on the way back we stopped in at Salisbury as we seem to be rapidly working our way round the cathedrals of England (this was the second of the weekend).

Caught a nice bit of setting winter sun through the branches of this tree in the cloisters, I wasn’t thrilled with the shot when I took it but have managed to clean it up quite a bit with Lightroom to the point I think its one of my favourite recent photographs.

 

All rights reserved

The sun in the cloister

Moving Git

I’ve been spending some time over the last month or so migrating our git server at work. Our old environment was starting to show the signs of age and just was not suited to having multiple developers checking in on several large projects at a time meaning we frequently ran into file permissions problems needing one of the team to ssh to the server a manually chmod the directory.

There are a lot of alternatives out there for git hosting including GitLab, Bonobo, Stash and the 800 pound guerilla of GitHub enterprise. In the end Stash from Atlassian won out, on premise hosting, great Jira integration and finally low license cost for small teams (which is in fact a donation to Room to Read).

Now we have a shiny new server all set up and running how best to migrate all our existing repositories over without losing history, tags, branches or living in a half and half state while we do so? One of the nicest things about distributed version control systems like git is that this process is much simplier than with large centralised version control systems of old, below are a couple of scripts for making this easier still.

(more…)

The other weekend a few old friends invited me down to our local paintball field, Campaign, to spend the day training with them. It was awesome to see the guys again and play some nice relaxed paintball. The new field Tony and his guys have built is truly impressive. For a few points we strapped the GoPro on to see how well it worked strapped to the barrel, other than showing just how much I need to practice the footage came out quite well.

Site Refresh

For the last month or so I have been playing with WordPress themes and bootstrap with the intent of finally replacing the slightly tweaked standard theme that I’ve been running for the last four years or so. It’s finally at a point where I am happy to leave it up permanently, there is definitely a couple of bits I need to tweak but as a whole I’m happy with how it has turned out. Bootstrap made working with dynamic re-sizing an absolute breeze, go on scale it up and down a couple of times.

The image above is just a part of one my favorite photographs. I took this at Reculver in Kent at sunset with my Sony NEX5. On the to-do list is a better photo plug-in so I can post a few more of these photos until I get that sorted here is a link to the full image.

Reculver at sunset

Reculver at sunset

If you fancy having a look around the theme code it’s all on GitHub.

Visual Studio 2012 marks a change in cycle for the development tools team as they are changing pace to incorporate regular updates to the product rather than the big bang of a new release every couple of years. Update 1 came out back in November (read more about that here) and I had been running it in the RC without issue. However with my fresh install of Express  I had forgotten this update, launching VS this morning to make a couple of changes I saw the update available reminder. Couple of quick clicks later the download is on its way. Code change done I fired off the update and all seems fine, progress bars, coffee, reboot. Reopen Visual Studio, open solution, none of the projects will load on hitting reload  for a single project and you are greeted with an alert with the text:

The composition produced a single composition error. The root cause is provided below. Review the CompositionException.Errors property for more detailed information.

1) No exports were found that match the constraint:
ContractName    Microsoft.VisualStudio.Shell.Interop.IVsHierarchy
RequiredTypeIdentity    Microsoft.VisualStudio.Shell.Interop.IVsHierarchy
RequiredCreationPolicy    Shared

Resulting in: Cannot set import ‘Microsoft.VisualStudio.Project.VS.Implementation.VSUnconfiguredProjectIntegrationService.IVsHierarchy (ContractName=”Microsoft.VisualStudio.Shell.Interop.IVsHierarchy”)’ on part ‘Microsoft.VisualStudio.Project.VS.Implementation.VSUnconfiguredProjectIntegrationService’.

Element: Microsoft.VisualStudio.Project.VS.Implementation.VSUnconfiguredProjectIntegrationService.IVsHierarchy (ContractName=”Microsoft.VisualStudio.Shell.Interop.IVsHierarchy”) –> Microsoft.VisualStudio.Project.VS.Implementation.VSUnconfiguredProjectIntegrationService

Resulting in: Cannot get export ‘Microsoft.VisualStudio.Project.VS.Implementation.VSUnconfiguredProjectIntegrationService (ContractName=”Microsoft.VisualStudio.Project.Designers.IVsUnconfiguredProjectIntegrationService”)’ from part ‘Microsoft.VisualStudio.Project.VS.Implementation.VSUnconfiguredProjectIntegrationService’.

Element: Microsoft.VisualStudio.Project.VS.Implementation.VSUnconfiguredProjectIntegrationService (ContractName=”Microsoft.VisualStudio.Project.Designers.IVsUnconfiguredProjectIntegrationService”) –> CachedAssemblyCatalog

Resulting in: Cannot set import ‘Microsoft.VisualStudio.Project.UnconfiguredProjectServices.vsUnconfiguredProjectIntegrationService (ContractName=”Microsoft.VisualStudio.Project.Designers.IVsUnconfiguredProjectIntegrationService”)’ on part ‘Microsoft.VisualStudio.Project.UnconfiguredProjectServices’.

Element: Microsoft.VisualStudio.Project.UnconfiguredProjectServices.vsUnconfiguredProjectIntegrationService (ContractName=”Microsoft.VisualStudio.Project.Designers.IVsUnconfiguredProjectIntegrationService”) –> Microsoft.VisualStudio.Project.UnconfiguredProjectServices

Resulting in: Cannot get export ‘Microsoft.VisualStudio.Project.UnconfiguredProjectServices (ContractName=”Microsoft.VisualStudio.Project.IUnconfiguredProjectServices”)’ from part ‘Microsoft.VisualStudio.Project.UnconfiguredProjectServices’.

Element: Microsoft.VisualStudio.Project.UnconfiguredProjectServices (ContractName=”Microsoft.VisualStudio.Project.IUnconfiguredProjectServices”) –> CachedAssemblyCatalog

That’s one very large dialog with just the less than helpful OK button. Visual Studio does occasionally glitch out but its been a while since I’ve seen one quite so spectacular and consistent.  A quick search for the issue above yields pages and pages of connect.microsoft.com reports of people facing the exact same issue, always with C++ projects always after update 1.

Visual Studio 2012 update 1 ctp 3 install has destroyed ability to load or create any project

Error on creating C class library in VS 2012 for desktop

Error when creating C class library project

So I’m not alone that’s always a good start, less good is that all are closed and none contain any answers. At this point I thought I’d check for more updates in Visual Studio, no joy automatic update checking is on but no update. Okay how about a repair? More progress bars, more coffee and another reboot later still the same error. Digging through the bug reports I found one report of user with a fix, wipe and reinstall OS.

After spending the best part of an hour searching, repairing and patching, Windows update kicked in on our schedule and prompted to check for yet more updates. Sitting right at the top of the list was KB2781514. Now that looks a little more helpful than a disk format.

KB2781514

The additional information doesn’t hint at a cause or even mention the issue shown in the short description above, instead it only states that it improves the stability of Visual Studio. With the patch applied normality is restored and projects once more load successfully.

It seems an unusual choice to offer the update pack of software via one mechanism but to offer patches by a different approach, especially one which may be tightly controlled within a corporate environment. It is a shame the sheer volume of reports on connect.microsoft.com prevent them from updating the report with the resolution when a cause has been found.

 

Absorbing books

I love reading. It’s that simple, I enjoy losing myself in a story or absorbing the experiences of someone else. However this year has been manic, between work, travel and restoring my new house I haven’t had many opportunities to just sit and read. However I realised the other day I have probably read more this last year than I have since I finished university. However the key to this is not always carving out time to dedicate to reading (though as I’m enjoying a week off as I write this I have grabbed a few hours of sitting with coffee and a book) it has been a change in how I absorb books. I’m using absorb in place of reading as not all my ‘reading’ has been purely looking at printed text.

I was lucky enough to be given a Kindle for Christmas last year and I have to say this has been the catalyst for the biggest change I have noticed. Reading has ceased to be purely a evening task I now grab a few pages at every chance I get. Between my actual Kindle device and my phone I always have the chance to slip in a couple of pages while I’m waiting for something. I have no more idle time, except when I actually want idle time. I recently travelled a week for work only to realise about as I stepped on the train my Kindle was sitting on my desk, never-mind whip out the phone synch my last read location and keep going. That week I finished an entire book on my phone as I was reading for a couple hours each night. Admittedly it wasn’t the greatest reading experience I much prefer the e-ink display of the Kindle, but I enjoyed a couple of hours each evening that would have otherwise been spent idly surfing the web or watching TV.

Earlier this year Audible finally released their long rumoured (and much delayed) Window Phone application, this had been the kick I’d been waiting for to resume my Audible membership. I’m on the basic tier, one audio book per month this covers pretty much any title in their collection. Audible is perfect for reading when you can absorb information but can’t sit and read a physical book. I used this on a recent trip where I did multiple six hour long drives, that’s a lot of reading time and it certainly beats listening to the radio or CDs. I think by the end of the project I’d listened to the entirety of Mud, Sweat and Tears by Bear Grylls and Every Second Counts by Lance Armstrong not a bad recovery of otherwise lost time. But my key use for Audible has been while working on my house, I don’t think it’s actually possible for me to paint without an audio book now. This has now resulted in an Audible purchase being as essential to a day spent painting as brushes and dust sheets.

There a still a handful of books that I still enjoy reading in paperback, these are mainly for the collection value. For example the Discworld series by Terry Pratchett, I have read just about everyone of these as physical books over the last few years and they are starting to look like a rather impressive set on my bookcase. I can’t bring myself to admit that halfway through I stopped and switched to ebooks it would ruin something of the collection aspect.

My reading list so far in 2012:

Kindle

Currently reading: Assassins Apprentice – Robin Hobb

Truckers – Terry Pratchett

Trojan Horse – Mark Russinovich

The 20% Doctrine – Ryan Tate

Cryptonomicon – Neal Stephenson

Snow Crash – Neal Stephenson

Whispers Under Ground – Ben Aaronovitch

American Gods – Neil Gaiman

Good Omens – Terry Pratchett & Neil Gaiman

Managing Humans – Michael Lopp

Can’t Swim, Can’t Ride, Can’t Run – Andy Holgate

Company – Max Barry

Liars & Outliers – Bruce Schneier

Born to Run – Chris McDougall

Neverwhere – Neil Gaiman

Public Parts – Jeff Jarvis

Moon over Soho – Ben Aaronovitch

A Dance With Dragons – George R. R. Martin

Audible:

Currently listening to: A Blink of the Screen – Terry Pratchett

Ready Player One – Ernest Cline

The Hunger Games – Suzanne Collins

Planet of the Apes – Pierre Boulle

The Long Earth – Terry Pratchett, Stephen Baxter 

The Invisible Man – H. G. Wells

Mud, Sweat and Tears – Bear Grylls

Every Second Counts – Lance Armstrong

Physical Books:

A Place of My Own – Michael Pollan

 Snuff – Terry Pratchett

PBTrainer on GitHub

I was playing with Window Phone 7 development for a proof of concept today at work which got me thinking about the PBTrainer project I wrote last year. Having a quick look on create.msdn.com told me that it is nearing the 600 download mark, not bad for an abandoned application. However there were lots of things I wasn’t happy with in v1.1; no infinite repetition sessions, poor about/versioning info and pretty ropey tombstone handling.

So having got a bit of a buzz today writing a quick app I dredged out the codebase and had a look. Ah issue number one it does not compile, looks like I was halfway through the v1.2 changes and got interrupted. Okay no issue I know what the changes I wanted to make were and making a few bug fixes would be an ideal way to get up to speed again. So lets just roll back to the v1.1 code base and start making changes. Issue number two I broke the cardinal rule, no source control. It’d only a little side project, only I will ever use it, I’ll probably rewrite most of this, all of these are fine reasons we tell ourselves not to bother with source control. Judging by the code comments and TODOs its living somewhere between v1.1 and v1.2 (probably where life intervened and cut the project short) so I’ll have to pick through the code and hope that I wrote good comments (and didn’t use the same excuse not to do those). So the first stage of the revival is complete I have corrected my mistake and committed the project to an public GitHub repository. Hopefully over the coming weeks I’ll be working my way towards v1.2 of PBTrainer, meanwhile I have learnt my lesson: whatever you’re writing source control first.

The London 2012 Olympics kicked off in style last night with a fantastic opening ceremony taking in the history of Britain, music and technology as well as the parade of the teams. This Olympics will be unprecedented in the amount of access the public gets to the athletes, Olympians have been sharing their training, their travel and will no doubt share their success and failure at events as well. But social media is allowing the public to be more involved than ever in spectacle events such as the Olympics, Twitter faced a heavy load last night as people tweeted their way through the opening ceremony giving an amazing back channel to the event.

But what were people talking about for the four hours? According to Twitter themselves the most talked about moment was Rowan Atkinson during Chariots of Fire. During the London riots last year I was playing with the Twitter API to see if it would be possible to extract key events from the noise of the crowd. Last night I was prompted to turn this on just to see what went by, now the algorithm it currently uses to determine key events is more than a bit flaky and requires training to an event. However what it does do very well is capture all of the messages it processes so they can be played back to train the algorithm. Last night I pulled down around 18,000 public messages from Twitter between 19:54 and 23:56 (I lost the last batch unfortunately as my machine crashed).There is bound to be a lot more traffic out there as with an API key you don’t get access to the raw twitter fire hose of tweets just whatever you can search for within your rate limit. However this gives us a good sample size to experiment with. So this morning I wrote a quick and dirty little util to run regular expression parses over these messages to find the talking points based on the key events of the night, here’s what I found:

After removing duplicates: 18670 unique messages.

TeamGB: 162 messages (1.11%)

The torch & cauldron: 169 messages (1.16%)

Steve Redgrave: 34 messages (0.23%)

Paul McCartney: 177 messages (1.21%)

Bradley Wiggins: 25 messages (0.17%)

The Queen: 972 messages (6.66%)

James Bond: 401 messages (2.74 messages)

#criticalmass (protest group): 720 messages (4.93%)

Mr Bean: 318 messages (2.18%)

Lord of the rings: 131 messages (0.90%)

Unmatched by any of the above: 11870 messages

You can see by that last number there a massive amount of data still in there to be categorized. At the moment my solution for identifying events is by  manually training a Bayesian filter (or as in this case picking key events from the running order), there’s a wealth of data mining tools available on the net and a range of more scalable methods to process this information. The problem is the speed at which this information needs to be processed, on Thursday night I attended a talk by Ed Parsons from Google who said big data isn’t a problem when it’s static but what we have now is a fire hose of information rushing past us. Twitter is a perfect medium for expressing what is happening right now, that is also the issue with trying to mine that data for geospatial display, it is only relevant for a short period of time.

I’ve published all the code to extract the matches onto GitHub along with the message set captured.