When big data meets Bob Dylan

I’ve been at two modelling and simulation conferences in the last month in which the subject of ‘Big Data’ has been discussed.  According to one speaker, a Professor, we are currently producing more bytes of data each year than were produced in the previous 5,000 – each year!

Now I’m good with numbers, but I lost count of the almost meaningless string of zero’s that represent this data explosion.  And so I mused, and wondered what Bob Dylan might have made of this data onslaught:

How many times must we tick the right box

before we’ve got what we need?

How many bytes must we store in the cloud

before we say ‘that’s enough’?

And how many ways must we cut what we’ve got

before our profiles are full?

The answer my friend is blowin’ in the wind

The answer is blowin’ in the wind.

Wiki describes the original Bob Dylan anthem as impenetrably ambiguous.  The answer is either so obvious it’s ‘in your face’, or so impenetrable that you’ll never work it out!  A bit like the wind.

But of course life is full of ambiguities, so this is not a plea to halt the headlong dash to capture more and more data.  There is plenty of evidence that this surge of data and our ability to derive real benefit from it is significant, particularly in healthcare.  It is, however, an appeal to avoid an over-reliance on Big Data as some sort of ‘magic bullet’ that will enable us to transform our health and care services beyond recognition.  I want to suggest 4 reasons why this is not the case:

1. Big data ‘individuates’ us.  But we’re more than that!  Our personal health outcomes and wellbeing are a function of our relationships; those who might care for us, and those we care for.  In recent work to identify risks of admission to care homes WSP recognised the vital contribution that carer support plays in continuing to care for someone at home, but matching datasets for an individual across health and social struggle to capture this vital element.
2. Big data cannot solve the integration challenge for as long as we train and employ people to function in over-specialised roles.  The recently published ‘Shape of Training’ report recognises this for the medical profession.  Other work previously undertaken by WSP in the area of Long Term Neurological Conditions also demonstrates the overlapping skills and competency frameworks necessary to inform workforce profiles for this group of patients.
3. Big data doesn’t guarantee our ability to deliver coordinated care.  In other work by WSP for the National End of Life Care Programme to evaluate an electronic system for the co-ordination of end of life care we found that the technology solutions could be trumped by the presence, or absence of a good relationships between disparate team members.
4. Big data must not be used to over-ride professional judgement.  In all of our simulation and modelling work WSP emphasises the aid that such tools provide to elicit and represent our knowledge in meaningful ways.  But all models, including the representation of complex data, are a product of our (limited) understanding.  They are wrong by definition, and it is in the environment of professional judgement, appropriately supported and safeguarded, that the ‘right’ decisions are made. 

The risk we face is in turning ‘big data’ into an entity in itself, a self-perpetuating depository ruled by layers of impenetrable formulae.  Such a beast, were it to be let loose unsupervised, would be blind to the realities of the world in which we live, with obvious consequences.

So, how many bytes must we store in the cloud?  The answer, my friend, is blowin’ in the wind.