UK Government’s single Website since 1994

I tweeted four screenshots a couple of days ago showing the main UK Government Websites since 1994. A few folks have asked for better quality resolution, so here are the best I currently have (if you have better, happy to share them).

I briefly covered the history of government attempts to duplicate the single Website / portal model of AOL and CompuServe in my piece Happy 20th anniversary online government.


1994 – Government Information Service


2001 – UKonline

2004 – Directgov

Gov.UK 2016.png

GOV.UK – 2012

Posted in digital, IT, IT strategy, public services, technology, technology policy | 1 Comment

King Canute, diffusion and the Investigatory Powers Bill


King Canute rebuking his advisors for suggesting he could hold back the waves

We can all learn something from King Canute. At least he had the humility to know, contrary to popular misconception, that he could not hold back the waves.

The same humility is absent however from the Investigatory Powers Bill – which seems to imply it can hold back waves of diffusion.

Diffusion is the way in which a new innovation, such as the car, television or the telephone, starts from a small niche market of early adopters through to being a commodity used by all. It’s best known from the classic Everett Rogers curve.

Think of any new innovation or technology – from flat screen TVs to contact lenses to electric cars – and it becomes apparent how diffusion works across multiple markets, industries and organisations.

Let’s consider how diffusion will impact the IP Bill by taking a look at encryption.

Computer encryption was initially limited to those with the compute power and expertise to use it – such as the intelligence agencies. Early work on public-key cryptography was kept secret. But as with any innovation it was only a matter of time before what was available to a very limited and select few followed the diffusion curve. Encryption moved out of its niche market and into the mainstream.

The impact of this process has been good for us all – better security for our online financial and commercial transactions, and better security for devices such as laptops and mobile phones. Successive waves of technical innovation have provided the intelligence agencies with short-term advantage. But over the longer term, those advantages flow out and diffuse to us all.

There’s a big downside too of course. This same pattern of diffusion happens in less helpful ways  – such as criminal hacking.

At one time hacking was limited to those with in-depth technical capabilities. Now hacking is increasingly commoditised. Today someone without any technical knowledge can download and run automated hacking scripts and launch potentially damaging criminal attacks without any real technical understanding. What was once niche and specialist has become mainstream.

And this is where diffusion and the IP Bill clash. Big-time. Here’s why.

The Bill talks about being able to demand “the removal of electronic protection applied by a relevant operator to any communication or data”. The Bill also seeks other significant powers, such as making it legally permissible to remotely hack computers.

So let’s assume the Bill passes. Someone creates a way to “remove electronic protection” from any communication or data. So too hacking tools are created and exploited so that computers can be remotely compromised and their contents accessed. So far so good – just what the Bill’s authors wanted. Trebles all round.

Ah. But we haven’t yet considered the impact of diffusion. Unfortunately what starts today as a specialist way of compromising security and enabling remote hacking will tomorrow become a commodity, available to all. A universal way to remove “electronic protection” from every device, communication or data.

It’s hard to believe anyone considers this a good idea. Consumers will no longer be able to trust their devices or online financial and commercial transactions, or businesses their mission critical information systems. Without trust, our online commerce and financial environment will fail. Worst of all, the intelligence and law enforcement communities will find their own operations and security progressively, and fatally, compromised.

The IP Bill in its current form will lead to the very opposite outcome to that its authors foresee.

More time is needed to get the Bill right. The wrong decisions now would prove devastating. Not just to our trust in technology – but to our personal and national security.

Just as King Canute accepted that he could not hold back the waves, the civil servants authoring the IP Bill need to recognise that they can’t hold back diffusion.

Posted in digital, future Britain, IT, IT strategy, privacy, public services, security, technology, technology policy | 1 Comment

security, privacy and the Internet of Things

photoI had the pleasure recently of opening a Cambridge Union Society debate on the topic of “This House Fears The Large Scale Collection Of Personal Data“. This theme is partly what inspired my CIO Column on “The Internet of Thieves“. The issue of enterprise and Internet security (or more usually in my experience the lack of security) has occupied much of my career – and unfortunately seems likely to continue to occupy much of the rest of it!


Jerry Fishenden proposing the motion at the Cambridge Union Society 2016.
Photo © Chris Williamson 2016.

Alongside me proposing the motion was Heather Brooke, the investigative journalist and freedom of information campaigner. And opposing us were solicitor and academic Professor Christopher Millard and journalist Edward Lucas. Both sides of the debate were complemented by a student speaker: supporting us with the proposition was Katherine Dunbar, student and competitive debater; and the opposers were supported by Katie Heard, Durham student and competitive debater – both of whom demonstrated their expert familiarity with the format and a commendable ability to have read and learned about the topic in remarkably short time.


Edward Lucas and Katie Heard consider their response, while Professor Millard looks on.
Photo © Chris Williamson 2016.

The formality of the setting and format of the debate all seemed a very long way from the “debates” we used to have at my old south London comprehensive. It made me realise how rarely I engage in formal debate – most conference events consist of “panel discussions” and tedious broadcast-mode slideware instead.

The core of my opening proposition was that far too many digital businesses rest on a profit model centred on the relentless commercial exploitation of our personal data. Some of our personal data of course we may share voluntarily with others in return for a benefit – store loyalty discounts for example. But a great deal are taken, analysed, manipulated, sold and exploited without our consent – often indeed without even our knowledge.


Professor Christopher Millard makes his case for opposing the motion. 
Photo © Chris Williamson 2016.

Not so long ago it was a unique, unpleasant characteristic of totalitarian states that citizens were permitted no secrets – no private, personal spaces. No freedom. We pointed wagging, righteous, critical fingers at such regimes.

So it’s ironic that a worryingly similar invasion of nearly every aspect of our personal lives has now been adopted as the routine, prevalent, business model of many Western companies and even, shamefully, some of our governments.

Let’s think about this another way. What would we make of somebody we discovered rummaging daily through our dustbins to examine our discarded letters, beer bottles and food packaging? Of somebody who stalks a few paces behind us everywhere we go to observe who we meet and to eavesdrop on and record our conversations?

We would, I think, regard such an individual to be perverted. Possibly even insane. Certainly not someone you’d invite to your birthday party – somebody probably best subjected to a restraining order.

unspecified 5.jpg

The debate in full flow at the Cambridge Union Society. 
Photo © Chris Williamson 2016.

And now imagine this person also randomly and obsessively runs up to us from time to time and shouts “I think you might want to buy this car!”, or “Are you looking for a new house?” or “You’re drinking too much!”.

Yet this invasive and obsessive behaviour is precisely how our technology behaves. Every second of every day. We should regard this use of technology – to trawl, monitor, gather and mine our personal data ­– as no less perverse.

Instead of using new technology to partner with us as equals to our mutual benefit, far too many organisations are obsessed with fleecing us of our personal data for short-term gain, without any regard for the consequences. All in the vainglorious hope that it will provide them with the power of precognition, the ability to understand us better than we understand ourselves – in order to take even more money from us.


Heather Brooke debating in favour of the proposition: “I believe in privacy for the private citizen going about their private business, and transparency for the public official making policy decisions which affect us all.”
Photo © Chris Williamson 2016.

But but but!“, my critics will counter, I am concerned for no good reason. We should all just enjoy the benefits bestowed on us by this largescale collection and use of our personal data. Where’s the harm?

Well, in response, consider the expert advice given by those who safeguard our critical national infrastructure. They warn of the grave risks of aggregating bulk personal data – creating a pool of valuable information that will be targeted, exploited and abused by everyone from foreign hostile powers to opportunist hackers.

We should heed such warnings.


Katie Heard opposing the motion.
Photo © Chris Williamson 2016.

If our bulk personal data is collected it will, without any doubt, sooner or later flow into the hands of whoever wants it. Whether by accident or design. So what? “Nothing to hide, nothing to fear.” Isn’t that what we keep being told? The same self-serving line trotted out by those totalitarian governments we once rightly criticised.

In any case, try parroting that nonsense phrase to a battered spouse, abused child, whistle-blower, informant, witness to a serious crime, journalist source, barrister and their client, or undercover law enforcement official. Do you really think they have “nothing to hide”? Of course they do, and for very good reason – this is part of the reality I was arguing in my column “Securing digital public services“. Access to personal data can, literally, become a matter of life and death.

This abuse of our personal data threatens us all in other ways too. It undermines our everyday security. What’s the point after all of protecting an online financial account with “secret” details of your first car, favourite colour and memorable place when those very same details are being Hoovered up and sprayed around the world?


Katherine Dunbar argues passionately in favour of the motion.
Photo © Chris Williamson 2016.

The irony is that all of this sucking up of our personal data isn’t even necessary: it’s the by-product of a badly broken and ill-conceived business model. How much simpler it would be if we had better business models, ones designed to enable and secure the Internet age. Empowering technology that lets us maintain and control our own personal data, and choose with whom we wish to share it.

What a terribly brilliant, but dangerous idea that is. Rather like democracy itself. Yet we urgently need to adopt this type of imaginative new approach if we are going to end the toxic legacy of analogue thinking in the digital age. The intrusive and dangerous large scale collection of our personal data needs to end, whether by businesses or ­governments. Our democratic right to safeguard and control our own personal data must be strengthened.

Until this happens, we must do everything in our power to protect our data – by using ad blockers, virtual private networks, cookie wipers, onion routing, end-to-end encryption. Whatever it takes to keep our data, and us, secure.

unspecified 6.jpg

Edward Lucas makes the closing case for voting against the proposition.
Photo © Chris Williamson 2016.

The large scale collection of our personal data must not be seen as some sort of ransom or blackmail we have to pay in order to enjoy the benefits of our digital age: quite the opposite in fact. I  supported the Society’s proposition that we should fear the current abuse of our personal data – because it has become the biggest risk to this emerging, amazing, exciting, digital age.

Reflecting on the debate afterwards, I think there was little significant distinction between either side – the underlying consensus seemed to be that we should all have better control over our own personal data. You can’t have security without privacy and vice versa.


… relaxing after the debate. 
Photo © Chris Williamson 2016.

The post-debate Press Release from the Society provides a high level summary of the debate, as does the short, edited highlights video below (I understand the “Director’s cut” full version will be available at the end of this academic term). This is an important topic that needs much more discussion and understanding – and not just in the debating hall of the Cambridge Union Society.


Posted in digital, future Britain, identity, IT, privacy, public services, security, technology, technology policy | Leave a comment

more CIO bits and pieces

Here’s a summary and links to my last few CIO columns of 2015. Looking forward to what 2016 brings.

Service design thinking

December 2015.

The use of technology in government is all too often relegated to simply making business as usual more efficient, or improving co-ordination between and across the arbitrary divisions of the public sector. Rarely has it managed to achieve the type of fundamental re-engineering of public services typical of modern digitally-enabled organisations, despite a long-held desire to do so. The real challenge is not technology – but to move from a bureaucracy-centric culture to a service-oriented one.

Securing digital public services

November 2015.

The IT reform agenda of the past five years has successfully demonstrated that government technology itself is rarely “special”. Public services can often be designed using commodity goods and services – much as in any other organisation. Yet one area where governments do face a unique challenge is in their duty to protect some of the most vulnerable and at risk individuals in our society. Government must resist calls to weaken cyber security if it is to continue protecting the most vulnerable and at risk in our digital society.

Posted in digital, future Britain, IT, IT strategy, open government, privacy, public services, security, social exclusion, social inclusion, technology, technology policy | 1 Comment

CIO roundup

Here’s a summary, with links to the full pieces, of some of my recent CIO columns:

Data-enabled service design

October 2015

The chorus of people calling for personal “data sharing” in the public sector seems to grow by the day. Yet rushing to propose “data sharing” is to start in the wrong place. If “data sharing” is the answer, what was the question?

Of lipstick and pigs in government

September 2015

The UK government is “in the vanguard of developing common IT architectures, and ahead of most in developing the IT core to enable secure transactions with citizens”. Is this part of the ongoing debate about the future of the Government Digital Service (GDS), GOV.UK and the role of “Government as a Platform”?

No. It’s from an international survey carried out in 2002, reflecting an earlier UK Government strategy of developing a cross-government infrastructure covering common services such as payments, authentication, transactions and secure messaging.

Time for the rise of the platform mutuals?

August 2015

Draw back the curtain of hype from many so-called “digital innovators” such as Uber and you reveal familiar pyramid shaped organisations that share many negative characteristics with the heyday of the railway and oil tycoons. Even claimed innovations such as “dynamic pricing” are merely shallow re-brandings of the economics of the barrow boy – putting up prices when something’s in high demand, reducing prices when it’s not. Yeah, very original. And, just like those earlier tycoons, these new businesses operate in a largely unregulated environment — beneficiaries, for now, of governments’ habitual failure to keep up with the times.

Re-establishing trust in technology

July 2015

It’s becoming difficult to remember what it felt like during those early, pioneering days of the internet. It seemed to hold out so much promise. I don’t mean the transactional convenience that dominates our usage today — online banking, booking holidays, music downloads, photos of kittens — but its more Reith-like ambitions. For a fleeting magical moment in time it held out the promise of new forms of political, social and personal expression and a more empowered, participative and inclusive society.

Then reality intervened.

Improving front line services

June 2015

Digital organisational practices enable us to make significant improvements to the way our public services work. But instead of seizing hold of this opportunity, many existing bureaucracies appear to place their own internal interests before front line services. There remains a widespread misunderstanding of “digital”, miscasting it as being about yet another generation of online versions of paper forms and processes rather than the catalyst for a wholesale reshaping and improvement of the public sector.

The result is damaging and unsustainable.

Posted in digital, future Britain, IT, IT strategy, open government, privacy, public services, security, social exclusion, social inclusion, technology, technology policy | 1 Comment

online citizen accounts

The idea of an online government account where we can see everything in one place has been kicking around since the late 1990s and the “” all-in-one portal. Despite several generations of government portals over the last 21 years (GIS, UKonline, Directgov, and now GOV.UK) we still don’t seem to be any closer to fulfilling that original vision.

HMRC’s online self-assessment portal for example looks almost identical to when it launched 14 years ago, in 2001, in that heady, optimistic flurry of early “e-government” services. Online government today still generally presents us with a set of silo transactions that mirror the paper-based processes that went before — despite the original plans to use technology to re-think and redesign government services.


Figure: HMRC’s self-assessment service some 14 years on

In this same time frame, even the high street banks have dragged themselves into the Internet age. They’ve gradually provided significantly improved services, through the use of PCs, smartphones and tablets, the deployment of chip and PIN cards and contactless payments, and quicker ways of transferring money electronically, such as Faster Payments. Partly as a result, total cash payments have been overtaken by non-cash payments.

In the best of the public sector we’ve seen organisations such as TfL (Transport for London) taking strategic advantage of these improvements to streamline and improve their own services, embracing the use of contactless payment for example.

Whilst the banks face similar challenges to government – including their complex brownfield IT estates with creaking mainframes in the back office – they’ve made meaningful progress in transforming their front-end operations. And even though the banks compete with each other, they’ve still managed to collaborate so that we can use our bank cards in any ATM and see our current account overdraft regardless of whether that machine is run by our bank, another bank or a third-party provider.

Making it happen – building on what’s in place

But just how accurate is this somewhat cynical view of how well government is handling technology to transform the way our public services operate? How hard would it be to now provide the type of online citizen tax account that the Chancellor mentioned in his budget? How far are we from realising that long-held vision of a more widely integrated online government account for citizens, and businesses, alike?

There’s been a lot of negative commentary questioning the ability of government to deliver the online tax service the Chancellor outlined. Much of the media commentary has focused on this becoming yet another project ripe for “government IT disaster” headlines. That it’s just a grandiose pipedream and will be far too complex to implement. Yet such understandable cynicism overlooks the infrastructure that the UK already has in place. The move towards the real time information (RTI) tax system implemented by HMRC over the last few years has already demonstrated how government can make much more rapid progress in delivering new services if it’s smart about how it works.

The implementation of RTI means that PAYE (Paye As You Earn) data now flows automatically into government. RTI’s success relies on the way in which it has aimed to integrate reporting obligations alongside the actual payments. For example, when an employer sends information about their employees’ salaries and deductions to government and simultaneously makes the actual salary payments to employees’ bank accounts, they fulfill all their obligations without the need for later reconciliation.

Whilst the interim, and incomplete, solution of RTI currently implemented by HMRC means in the short-term that employers haven’t been able to completely integrate payment and reporting, it does give a promising pointer for the future.

RTI interim

HMRC’s real time information (RTI), simplified view of current (interim) design

As the figure above shows, at present RTI data currently flows through two separate channels. For the 70,000 largest submitters, these two processes are automatically joined up by means of a “BACS hash”. This is a convoluted but workable solution that enables payments made over the BACS system to be verified and matched to employers’ payment declarations. This less than ideal interim solution was adopted largely because of representations from the payroll industry, who expressed concerns about changing both what was sent and the channel it was sent over. It’s intended as a staging post on route to the original fully integrated system.

The full solution for RTI envisages a much more streamlined re-use of the existing banking infrastructure. The UK’s banking network processes around 10 billion transactions each year (about ten times the volume of transactions that HMRC handles), with a combined value of about £5 trillion. It provides the central infrastructure for BACS Direct Debit and Credit payments and the Faster Payments Service, and connects the world’s busiest ATM network of over 69,000 machines.

The UK payments industry has already recognised that this existing central infrastructure easily has the capacity to carry the extra data required to meet government’s requirements (a meagre 18 characters per transaction). It has therefore publicly announced its commitment to work with the UK government to develop a strategic, complete solution that would replace the interim RTI phase — an initiative now nicknamed “Richer Data”.

HMRC richer data

The proposed “Richer Data” approach (the original strategic design for RTI)

The significance of these developments goes well beyond PAYE. Although the above Figure illustrates a payroll payment, the infrastructure would enable any payments and the information associated with them to flow regardless of the type of financial transaction. The architecture will enable not only transactions of interest to government, but will also enable businesses to include relevant data with other forms of payment, helping them rationalise and automate many of their other business processes.

Given these existing developments, the next logical step would be to enable citizens (and potentially businesses too) to login online to see and manage their data for themselves. Over the year we would be able to see in near real-time everything we have been paid as employees and all the taxes deducted. It would help to provide the type of experience we already have grown accustomed to with online banking, where we can keep track of our finances in near real time.

So the first step towards providing us with online government accounts should not be “a major IT project”, but a programme that enables us to access our existing data based on enhancements to the UK’s core national payments infrastructure — a programme not run as a “government IT project”, but a joint programme in partnership with those who currently own and run it.

HMRC could surface this information via their own portal, where they already provide other HMRC services, elsewhere on the GOV.UK infrastructure, or through other access channels. For logging in to such a government-based service, there’s already the old Government Gateway authentication system, although any new services are likely to use the replacement Verify identity assurance system to enable us to login as securely and easily as possible. Equally, another option would be for us to access this data through our existing secure, trusted and familiar online banking services — via online banking services for example, and to view or print the data at an ATM. It will be interesting to see whether we are given a choice of channels as plans for the citizen account develop.

Making it worthwhile

By making smart use of the UK’s existing payments infrastructure, within a fairly short time period we could have an online service that lets us see everything we earn and everything we have contributed to the government from an employee perspective. This would be a useful first step. We would be able to see in near real time our current year’s contributions and earnings and, over the years, we would begin to build up an historic record of our cumulative contributions and earnings. So far so good.

But this system would not provide a complete picture. Any other earnings, such as from savings, that do not go through PAYE would not appear. And our relationship with the state is not one-sided: wouldn’t it be useful to see any benefits or welfare payments being made to us too? After all, one of the other reasons for the implementation of RTI was to provide up-to-date information about employment and pension income so that the Department for Work and Pensions (DWP) can determine and adjust claimants’ Universal Credit awards.

What we ideally need is not a partial set of data, but an equivalent to our online bank accounts, showing monies in and monies out. It wouldn’t be much use only having part of the picture, of seeing what we pay into the system without the balancing information about how much we benefit too.

Surprisingly, enabling us to access this additional information needn’t be such a big subsequent step. The banks already send to HMRC the taxes deducted from our savings accounts. So this information could also be rolled into our citizen account. On the welfare side, DWP uses a duplicate of the same data gathered by RTI to help inform the calculation and determination processes that decide our entitlement to Universal Credit payments. So this data could be made visible in our online accounts too.

Given that much of the infrastructure and data already exist, in a fairly short space of time we could have an online service that enables us to view all of our earnings, our payments into government and our payments received from government. Over time, this would also build up into a useful lifetime record of our earnings, contributions to the state and receipts from the state. A true citizen account rather than only a partial and incomplete view.

citizen account mockup

A simple mockup of an initial citizen online account

Extending the model

Would this be enough? It would certainly be useful, but in some aspects it would also still be incomplete. What about other taxes, such as indirect taxation like VAT? For most of us, the tax we contribute via indirect taxes such as VAT will also be significant on an annual basis — as are other taxes or duties if we fly, drive, smoke or drink. It would be good to see all of this too, but how feasible would that be?

Not as difficult as you might think.

Part of the way that the “Richer Data” initiative will work is via the use of a financial data standard (likely to be based on an open standard such as ISO 20022). This would enable additional data to be carried alongside financial information – such as the additional payroll data that now flows with RTI.

For example, suppose we pay £12.00 using a credit card in a shop, which is actually £10 + £2 VAT. At the moment, when we receive our credit card bills we don’t see this breakdown, merely the gross amounts including tax and the total amount due: we lose insight into the amount of tax incurred in our daily lives. However, if these transactions used the ISO 20022 standard, both our credit card statements and the online citizen portal could reflect this breakdown, not just the total amount, as at present. It would enable us to keep track automatically of our VAT contributions. Indeed, it could conceivably cover all national or local government payments – council tax, parking and congestion charges, even library fines and prescription charges.

Given that many of these transactions already take place over the banking network, capturing this information could be automated using the same processes and data standards. In the same way, if we pay for an evening out in a pub or fill up our car with petrol or pay for a holiday flight, the data captured and shared back to us could also include other related taxes, such as beer, petrol and air passenger duty.

new payments infrastructure

How payments data could flow from e.g. retail outlets

This approach would enable the much-promised online citizen account to become a rich resource to us in terms of our interactions with the state. Yet a single “citizen portal” should not be the only option. As with the current ATM system, which lets us see our account balance anywhere we choose (even at a competing bank’s ATM) we should also have choice about what channels we can use to see and track our information. It shouldn’t just be accessible from a single government website: that would take us back in time to the sort of top-down design and massive, monolithic “government system” thinking typical of the late 1990s.

But whoah! Hold on a moment – won’t this system I’m describing also enable the state and our financial providers to know far too much about us in terms of where we shop, what we buy, how much we drink and smoke and so on?

Making it private and secure

Whilst a data rich online citizen account could be enabled relatively simply, and in an incremental fashion that avoids the “big bang” chaos of some previous government programmes, clearly there are significant privacy and security concerns that must be addressed.

Would we be happy for all of this information to be gathered and stored in a single place? It would be incredibly valuable data in the wrong hands, providing rich insight into many aspects of our private lives, where we spend and on what, and how dependent we are on the state. Inappropriately accessed and used, it would be a potentially toxic resource and effectively function as a confessional self-reporting system on where and how we live our daily private lives.

If we are to have a useful online citizen account, security and privacy need to be built into the system by design. It should aspire to comply with the sort of principles Kim Cameron set out in his “Laws of Identity”. The system must avoid the inadequate technical design of earlier government initiatives, such as the national identity register and its associated identity cards which based themselves on a simplistic model from the 1930s. Such systems demonstrated poor systems design and engineering, neglecting to use modern technologies that provide stronger data protection and hence enhanced levels of security and privacy.

It’s been possible, for example, to conduct a transaction such as confirming someone is a higher rate taxpayer or in receipt of child benefit without revealing anything else about them since at least the early 2000s [1]. Such techniques need to become commonplace rather than the sloppy “data sharing” approaches which assume personal data needs to be copied and shared everywhere, with the inevitable leaks and abuses reported so frequently by the media. Newer technology options, such as homomorphic encryption [2], blockchain [3],  certificate transparency [4] and ‘Guardtime’ [5] should also be robustly evaluated to see what potential role they could play in a secure, privacy aware and citizen-centric service.

If government applies good privacy and security engineering to an online citizen account, it would also have beneficial impacts on the wider financial system. It would raise the bar for example on the security used within banking and retail operations, which remains relatively leaky today. They too could move to take advantage of the more secure technologies that an online citizen account will require.

Such a system would also need to give us control over our personal data (in line with government policy).

government policy

Government policy on citizens’ personal data (source: Government Service Design Manual)

Better than that, it must also be engineered so that it is impossible for the system to hold inappropriate detail or enable anyone to reverse engineer our interactions without our explicit consent (or under due process of law). That is why ensuring the right technologies are engineered into the design before it is developed is essential. This would enable transactions to be verified and authenticated, and for data to answer questions such as “Is this person entitled to a tax credit?” but without any intrusive “panoptic” central authority holding all the details.

Such a system must also enable citizens to continue to use cash where they wish, but to obtain a point of sale receipt that enables them to manually enter records if they want to keep track of all interactions, including those that don’t use digital technology. (For insight into the potential dubious consequences of moving to a completely cashless society, this article is worth a read).

Many of us already interact with what is regarded as a generally safe, secure and ubiquitous financial system that we feed data into. After all, using online payments we’re becoming accustomed to services that let us transfer our hard-earned cash out of our own accounts and to a sequence of numbers that we trust to be the account of our intended beneficiary. We must be able to trust this system to protect our data and use it for the purposes we designate if we are going to increase our dependency upon it even further.

So all the components and technologies are in place, or in near-development, to make this vision a reality. A step-by-step approach can build on what is already there. It now requires a clear political commitment to ensure the whole design remains secure and privacy-friendly. The design of such a system must be done in the open, so that the UK’s expertise in privacy and security engineering can contribute, helping to review and improve the design. Building on the Cabinet Office sponsored private/public open collaboration on identity, represented by the OIX open forum, we also need a payment equivalent where payments standards and interoperability can be developed to meet these requirements, but with sufficient commercial incentives to maintain a competitive and innovative drive.

Where some of the necessary security and privacy technology is not quite ready for prime-time, government should play an essential catalysing role in encouraging further research and development – another good reason for working on this in the open. Not only will the citizen account then become an exemplar of how to create modern, secure and private citizen services – but it will also provide a unique competitive advantage to the UK, helping researchers, public services, and commercial and financial businesses to develop world-leading secure online computing and personal data models, technologies and products that will be in high global demand.

Wider benefits

There should be many other beneficial and far-reaching side effects to engineering the online citizen account well. For example, it would become simpler in real time to see other data, such as which major businesses are being subsidised by government – where employees are in receipt of tax credits to top up their low wages for example. Such information should also be easily accessible in the public domain.

insight into business data

Visibility of which businesses receive taxpayer subsidies

Such transparency would help inform the debate about the extent to which taxpayers subsidise apparently profitable businesses and enable better modelling of things like whether increasing the minimum wage to the living wage would produce a better outcome for all than business subsidies made via tax credits. Equally, it might be appropriate for government to consider reclaiming such taxpayer subsidies from a business’s annual profits. Without accurate and complete data, many such policy considerations are little more than a stab in the dark at the moment – but the moves towards a well designed system, with the right policy and engineering safeguards built in, would provide far wider benefits than just the immediate citizen account itself.

There are also likely to be considerable knock-on benefits to government’s own operations. It would make it possible, for example, to decide whether to continue to maintain two separate organisations with many duplicated functions, one which takes money from us (HMRC) and another which gives it back (DWP), with all the costs and friction (and citizen inconvenience and personal hardship) this artificial split can currently create. Given the type of data that RTI already collects and the type of data that would be available in a citizen account, it would be much simpler in future to have a single set of calculations that offset both deductions (taxes) and allowances (welfare). Doing so could improve the services we receive, cut their operational costs and inconvenience, and reduce the levels of fraud in the system.

The result is that government will progressively be able to streamline its own processes and organisation to better meet the needs of citizens and businesses, providing the type of digital transformation centred on improved public service design that we discuss in our book “Digitizing Government”. For the majority of citizens and businesses the administrative burden and frictional and human costs would reduce and government could better focus its efforts on those whose need more support and a helping hand — as well as homing in on those who intentionally fail to comply and contribute to our society.

Such a system should be incrementally developed, proven and improved over time rather than trying to do too much all at once. For example, we receive many other benefits from the state that are harder to quantify – from education to healthcare, policing to defence, and from road building to public transport. Working out how to determine the shared benefits we take from these will remain a much more complex challenge. But the route for the journey ahead is clear to see.

What we need now is a much better informed public debate about how such a system can be made to work in the best possible interests of us all. After all, this is not just about delivering another “government technology project”, but about addressing wider societal and policy issues too.


[1] See for example some of the techniques developed by Stefan Brands and his former company Credentica here.

[2] Homomorphic encryption

Homomorphic encryption is a form of encryption that allows calculations to be carried out on encrypted data. It generates a result that is the same as if the data had not been encrypted. This means, for example, that it is possible to perform calculations on financial data that has been encrypted without revealing the actual data. These characteristics make it useful for deployment in a system where a considerable amount of personal data is collected in one place – such as the proposed citizen online portal. It would potentially enable data to be encrypted and hence inaccessible to anyone except the owner (i.e. the citizen) but would still enable e.g. HMRC or banks to perform calculations on that data. As with blockchain technology (see below), it is not yet fully mature – but another area where government can play an important role in helping drive its development and adoption. You can read a bit more about it in this “American Scientist” article.

[3] Blockchain

A blockchain is a public ledger of all transactions that have ever been executed. It provides proof of all the transactions on a network and a full history of transactions. Transactions are entered chronologically in a blockchain and the blockchain database is shared by all nodes participating in a system so that no single node can ever be in the position of falsifying or tampering with it. The full copy of the blockchain has records of every transaction ever executed. It could be used to ensure that our transactions have happened and cannot be tampered with, yet it can also potentially retain a degree of anonymity – which is why it has provoked such interest with Bitcoin, the digital currency, since it enables secure financial transactions to take place without necessarily revealing who is involved in those transactions.

Within a system using blockchain technology, users can be identified only by their public keys. The mapping of a user to their public keys is held on that user’s node only and each user can generate as many public keys as they want, using each in a different context (such as a transaction with a particular retailer), and potentially also using one-time public keys to further reduce the risk of anonymity being compromised.These characteristics – proof that a transaction, such as a purchase or payment of tax, has happened, combined with potential anonymity – make it a candidate technology to be considered for use in a twenty-first century system. However, the ability to maintain anonymity may require better design than that currently used in the Bitcoin network, as this paper (PDF) points out, and its vulnerability to concerted manipulation by an adversary with sufficient computational power remain a concern. This piece is also interesting about its current limitations and (possible) future direction.

[4] Certificate Transparency

This technology is centred on a public, verifiable, append-only log — see Ben Laurie’s 2014 post here.

[5] Guardtime

This technology provides real time detection and mitigation of integrity loss in network infrastructure. It aims to combat cyberattack and data breaches through the use of a blockchain-based digital signature system for real time validation of electronic data. See this PC Advisor article and this Wikipedia entry.

Posted in digital, future Britain, identity, IT, IT strategy, open government, privacy, public services, security, social exclusion, social inclusion, taxation, technology, technology policy | 1 Comment

A (brief) history of UK Government moves towards a platform-based architecture

The vision — 1998 onwards

There’s been a long-standing vision to use technology in a way that doesn’t fossilise the old paper-based way of running the public sector — but which enables services to be redesigned and improved. Part of this vision has seen a move over the last few decades towards common services that remove duplication and enable the delivery of services designed around the citizen (rather than the owning department or agency).

The Parliamentary Office of Science and Technology (POST) back in 1998 provides the first recorded effort I’ve come across that considers the impact of technology on government and all of its services in a more holistic way. Their report on “electronic government” included the following schematic as part of a discussion about common processes across government and the need to avoid merely using technology to serve up online versions of paper forms.

POST gubbins

Figure: the 1998 view of common services

[Source: POST Electronic Government 1998]

The UK government’s architectural approach to development of these more joined-up services was set out in the 1999 “Portal Feasibility Study”, prepared for the Central IT Unit (CITU) in the Cabinet Office. It proposed the three-tier architecture shown below.

3 tier conceptual architecture

Figure: The 3-tier conceptual architecture, 1999

[Source: Portal Feasibility Study]

This architecture was designed to support a wide range of access channels, and to enable the existing IT estates of departments and agencies to integrate without requiring a big bang, rip and replace approach. It proposed that

A three-tier architecture can be used to insulate the access channels from the complexity of the Government Back Office with web technology providing the portal, or gateway between the channels and the individual service requested. The key concept of the three tier architecture is the use of middleware technology to provide a brokerage capability, a concept that sits well with the idea of a portal. The middleware will link components to allow them to interact without the need to have knowledge of the other component’s location, hardware platform, or implementation technology. [p.7]

 It included an emphasis on the value of open standards:

The technical implementation of the three-tier architecture must provide the glue to link existing Departmental services and systems to a wide range of different access channel technologies. This means that open standards need to be proscribed and that the interface standards needed to ensure good interworking must be defined.

An open architecture will maximise the flexibility and opportunities for infrastructure provider competition. Every major interface in the architecture will need to have an interface specification defined for it. This will allow architectural components, services and supplier systems to be replaced easily and a ‘plug and play’approach to be taken to architecture components, services and supplier systems. [p.8]

For the specific standards, reflecting some of the dominant technical models of the time, it recommended:

Tier Element Recommendation
Front Web server protocol HTTP
Application server candidates CORBA ORB (supporting services written in JAVA, or DCOM and Microsoft Transaction Server)
Middle Application server architecture candidates CORBA ORB
Back Suggested protocol for communication with Government departments SMTP
Suggested messaging standard S/MIME

Table: Proposed Technical Standards

[Adapted from “Portal Feasibility Study, 1999”]

Confidentiality was to be assured through the use of encryption between the desktop and the Web server, specifically SSL or HTTP/S. Public Key Infrastructure (PKI) would be used in the middle tier, with a firewall deployed to limit network access between the front and middle tiers. Messaging between the middle and back tiers would use structured messages digitally signed by the Web site to: protect the integrity of the information, authenticate the source and prevent repudiation. The internal government networks (the Government Secure Intranet, GSI, and xGSI) would be used to secure the physical transmissions within government’s own estates.

From concept to engineering

The approach — building out a single online web presence for the whole of government —  outlined in the “Portal Feasibility Study” was later to be iterated through a series of implementations. The initial site was “UKonline”, replacing the early portal efforts of the Government Information Service (GIS) at first established in 1994. This appeared as a soft launch, beta site in November 2000, providing time to gather feedback and enable refinement and improvements prior to its formal launch in February 2001. It was redesigned, rebranded and relaunched on 1st March 2004 as Directgov. The current GOV.UK first appeared in May 2011 and went officially into full service in October 2012.

However, my focus in this brief blog is not the front-end website but a range of other, associated platform components, such as those developed to secure the processing of transactions and to enable the reliable identification, verification and authorisation of citizens, businesses and intermediaries.

From late 2000 onwards, the UK government built out a series of web-service / SOA-based components. There was a clear business case for taking this “whole of government” approach: it was already evident that without some degree of common services, each part of the public sector would proceed with its own local initiatives — from websites to identity systems to payments systems and so on. The same needs would be met time and again in multiple places, resulting in pointlessly duplicated expenditure, processes and systems.

This would not only be expensive, and duplicate common infrastructure in multiple places across the sector, but also produce a poor user experience:  government services would be scattered across a technological landscape that replicated the physical world of multiple overlapping providers. Leaving everyone to do their own thing would effectively fossilise existing, paper-based services in the online world rather than taking the opportunity to improve and streamline them.

Open standards were the bedrock on which these initiatives were based, using published APIs (for programmatic access) and XML (for data interchange). Although collectively referred to as the “Government Gateway” (reflecting terminology first used in the “Portal Feasibility Study”), the initial component services consisted of several discrete functional components: 

  1. Registration & Enrolment (the identification, authentication and verification of online users — citizens, businesses and intermediaries — utilising a range of credentials, from user ID and passwords to digital certificates to — later — EMV chip and PIN cards)
  2. Transaction Engine (handling the processing of transactions between citizens, businesses and departments, including the management of state and orchestration across multiple service providers where a transaction involved multiple entities)
  3. Secure Messaging (handling two-way secure communication between departments and citizens)
  4. Payments Engine (handling in-bound payments to government departments)
  5. Gateway Helpdesk (an internal service enabling departments to integrate a degree of management of these common components services within their existing helpdesk services)

These API-based components are illustrated at a high level below.


Figure: the API-based components developed from late 2000 onwards

The rationale for adopting this cross-government approach has been explained as follows:

“The Gateway was designed to simplify and accelerate the UK e-Government programme. It achieves this by ensuring that the common building-block components of e-Government services are provided once, in a flexible, modular and scalable way.”

[Source: “Delivering e-Government Services to Citizens and Businesses: The Government Gateway Concept”. Jan Sebek, p.127. Published in “Electronic Government: Second International Conference, EGOV 2003, Volume 2”. Editor Roland Traunmüller]

The services were exposed via the following open standards:

  • TCP/IP
  • HTTP
  • HTTP 128 bit SSL (TLS 1.0)
  • HTML
  • XML
  • X.509 certificates
  • W3C XML digital signatures
  • SOAP
  • SMTP
  • SAML

The schematic below shows the relationship between these initial component services and also illustrates how the provision of common platform components avoids duplication of the same needs across multiple government services. It came about, in part, because each department was developing its own local solutions to its online service needs, with a resulting duplication of expenditure and fragmentation of services from a user perspective. Moving to a strategic, platform-based approach for common services made sense from an economic, architectural and user experience perspective.


[Source: “Delivering e-Government Services to Citizens and Businesses: The Government Gateway Concept”. Jan Sebek, pp.125-128. Published in “Electronic Government: Second International Conference, EGOV 2003, Volume 2”. Editor Roland Traunmüller]

All of the technical standards used were stipulated by government to be open, enabling interoperability regardless of the software or vendors involved across the many connecting organisations:

“A wide range of systems have interoperated with the Government Gateway since its launch, including systems running Sun’s J2EE technology, IBM technologies, Apache, Tomcat and other technologies and applications including standalone PC application software.”

[Source: Preliminary Study on Mutual Recognition of eSignatures for eGovernment applications NATIONAL PROFILE UK April 2007. p.16. IDABC European e-Government Services.]

Building on the Platform

Following on from the successful release and implementation of the initial cross-government components, in 2003 a further set of services were considered.

2003 components

Some initial work was done to prototype various of these additional platforms, but internal issues relating to funding and agreement between the multiple stakeholders involved became increasingly problematic. The recurrent issue of how much government should itself try to build in-house and what it should consume or purchase from others was also as much a debating point then as it is now. Hopefully this time around the debate about continuing to develop the vision of government at the technical platform level will be informed by the use of techniques such as Wardley mapping, which can help identify when and where in their evolution the various needs are and therefore how they might best be sourced (from bespoke build to rental).


 Figure: Wardley Mapping 

[courtesy Simon Wardley]

For those few who have not seen it yet, Mark Foden’s short but useful video on “the Gubbins of Government” is essential viewing. It sums up much of what has been intended and what remains to be done very succinctly to implement the platform model at scale:

Current era

In 2010, “Better for Less”, which set out many of the ideas implemented in government during the 2010-2015 period, emphasised the value of the component-based approach.

The use of a common infrastructure is critical … applications must be developed to work together using open standards for data and interoperability. It is critical that the common infrastructure is available for all the identified stakeholders: officials, citizens, third sector organisations, potential and actual solution providers …

… Government IT is well placed … if it adopts this construct: service oriented architecture, open standards, re-use of components and a carefully thought-through incentivisation environment encouraging and supporting innovation and re-use. It can take advantage of a cheap, commodity platform, competitive suppliers, constant innovation, and automatic sustainability.

In 2013, GDS published in its Service Design Manual its intention to continue this journey towards “Government as a Platform”. The more detailed text available at the time appears to have been removed from the site, but is still viewable in GitHub here. It recognised the need to work smartly including:

A move to platforms does not mean that government has to develop everything in-house: many of government’s needs can be met by existing cost-efficient utility services. However, government can help to establish best practice in areas such as personal data privacy.

Wherever appropriate, the government should use existing external platforms, such as payments services (ranging from third party merchant acquirer services to the UK’s national payments infrastructure). Deciding to develop platforms in-house will happen only where that is the best way to meet users’ needs in the most flexible and cost-effective way.

Government will draw upon the experience of other organisations that have already made this journey. Many have created platforms that initially sit across the top of existing silos to expedite agile and effective digital service delivery.

This co-existence enables the benefits of the platform model to be realised quickly, even in a brownfield environment such as government, while the silos below the waterline are gradually reduced in importance and eventually made obsolete.

This is the approach that government will follow, ensuring that it develops a well-defined schedule for switching off legacy environments as the platform model is progressively implemented.

Another announcement about this commitment was made in September 2014 by Sir Jeremy Heywood, Cabinet Secretary and Head of the Civil Service, in a blog entitled “More than just websites“. It seems clear that the continuing move towards a platform-based model is now generally accepted not only at the engineering level but, as importantly, at the business and services level too.

Where next?

We shall see after the election:-)

Technical Annex

I originally planned to include the following more technical detail in the body of my blog. However, it’s a bit distracting for those only interested in the history of efforts to realise the higher level principles and benefits of developing a platform based model. But for those who want to look under the bonnet in a bit more technical detail, here it is … It aims to provide insight into two earlier components developed from late 2000 onwards — the Transaction Engine, and Registration and Enrolment.

Transaction Engine

The Transaction Engine (TxE) was developed to provide a transaction handling and routing service. It supports the exchange of data (usually in the form of documents and business forms) between government organisations and external organisations, intermediaries and citizens, as well as between government organisations. It’s able to manage state between both single point to single point, as well as to orchestrate more complex interactions involving multiple parties where state needs to be understood and managed (e.g. an incoming piece of data may successfully update 3 of 4 departmental systems, but not the remaining one: clear state-management rules can be applied in such circumstances, ranging from rolling back all of the updates through to providing response information that indicates which systems have been updated and which have not and leaving the client side to decide how to progress the interaction).

TxE provides two main methods of utilisation:

  1. for ‘ad-hoc’ use, a defined submission protocol involving the exchange of XML documents posted into a URL destination point
  1. for permanently connected government and related entities, two-way messaging via Departmental Integration Servers (DIS) between the hub and those spokes involved in the messaging exchange

Although the original TxE was based on proprietary technology, the protocols developed and used were vendor neutral, open standards-based rather than utilising Microsoft’s BizTalk Framework. The development of these platform components were managed within the context of the broader e-Government Interoperability Framework (e-GIF) and its associated GovTalk initiative to use open interoperability standards (which drew primarly upon the IETF, W3C and WS-I interoperability standards).

XML was used as the standard data format for all messages into and through the TxE. The specific standards utilised were defined by the former GovTalk initiative. This is now defunct, with the UK Government’s Open Standards Board taking renewed and reinvigorated ownership of the open standards to be used in government for software interoperability, data and document formats.

Interoperability between the TxE and existing departments is accomplished through what became known as the “Departmental Interface Server” (DIS). DIS provides SOAP reliable two-way communication between the connecting organisation and the TxE. It combines compliance with TxE’s open standards (XML, HTTP and SOAP) together with reliable once-only delivery and separation of the integration requirements of the systems within the connecting organisations.


Figure: Standards

[Source: Government Gateway technical documentation]

Some departments and other organisations deployed DIS services developed by third parties rather than using the Microsoft Biztalk-based offering: SoftwareAG for example supplied HMRC and others, demonstrating the practical benefits of open standards in maintaining an open market rather than restricting it to a single supplier.

TxE protocols

Three message protocols are supported by the Transaction Engine:

  • Document Submission Protocol: the interface used by ISV Applications and Department Portals. It allows ISV applications or Portals to submit business transactions/forms/documents to Department services
  • Hub and Spoke Interchange Protocol: used for Department to Department business transactions/forms/documents submission
  • Administration Message Protocol: an asynchronous message based version of a more recent SOAP Admin Interface

GovTalk Message Envelope and Document Submission Protocol

DIS uses the GovTalk Message Envelope in conjunction with the Document Submission Protocol (DSP) from 2008. The DSP routes business transactions (e.g. Self Assessment tax forms) submitted from either a Department Portal (e.g. the HMRC Online service Web site) or directly from an ISV application (which could be hosted on a website, server-based, PC-based, smartphone-based etc.), through the TxE, to the appropriate Department (back-end) system and retrieves the corresponding response. The TxE acts as a “hub” to numerous “spokes”, which can be websites or dedicated DIS endpoints that provide onward integration and interoperability into departmental local systems.

DSP uses the GovTalk Message Envelope to encapsulate business transaction documents. GovTalk documents are XML formatted and use the UTF-8 encoding standard. Messages are transported using the Hypertext Transport Protocol (HTTP). Portals, ISV applications and Departments (using DIS) must be capable of generating HTTP 1.1 POST requests and receiving and interpreting HTTP 1.1 response messages.

The transaction architecture is componentised, with applications and portals separated from the systems with which they interact. So, for example, HMRC’s portal does not talk directly with its backend, but via the DSP and GovTalk Message Envelope, routed and authenticated via the TxE. Connections with the TxE are made either over the internet or via government networks (formerly the GSI, now the PSN).


Figure: end-to-end overview of the transaction-handling system

[Source: Government Gateway technical documentation]

The typical sequence followed when a client application submits a document to a target spoke (i.e. a Department service) is shown below (assumes no errors).


Figure: overview of the submission-response protocols

[Source: Government Gateway technical documentation]

The protocol makes extensive use of the envelope portion of the GovTalk schema, requiring XML documents submitted to TxE to include a Qualifier element immediately after the Class element. Together these two elements denote the message type.

Messages issued by the client application:


Messages issued by the Gateway: 

  • SUBMISSION_ERROR – error detected in message received from the Client
  • SUBMISSION_RESPONSE/ERROR – business response/error from Target Spoke (i.e. Department)

When submitting any message type to TxE it’s the responsibility of the client to ensure each message conforms to the relevant syntactical rules for that particular type of message. A client application does not necessarily have to process each document sequentially as described above. Instead it could operate in a batch mode, submitting a number of documents over a period of time and then later using: 

  • DATA_REQUEST to examine the state of these submissions
  • SUBMISSION_POLL to retrieve the corresponding response for each submission
  • DELETE_REQUEST to delete each submission from the Gateway
  • SUBMISSION_REQUEST to resubmit documents with recoverable errors

Registration and Enrolment (R&E)

The registration and enrolment (R&E) service was launched in January 2001. The initial authentication and authorisation component supported two credential types, UserID/Password combinations and digital certificates. The service continued to be enhanced after launch and later expanded to include WS-Security tokens (offering the potential for other trusted issuers to federate their tokens) and the use of EMV Chip and PIN card (from around 2008, offering the potential for users to authenticate using a third party card, such as one issued by their bank).

Citizens, businesses and intermediaries were therefore able to have single sign-on facilities across all government services — national, regional and local — although the option also existed (should a citizen wish) to utilise a separate method of authentication to each separate government service. This was to enable citizens to maintain the segmentation of their identities across government should they desire to do so.

Part of the problem tackled by this common service is the need to associate an online identity and its associated credential with the different identifiers by which that same user is known within different parts of government.


Figure: mapping an identifier to other known identifiers

[Source: UK Government Gateway Frequently Asked Questions 2005] 

Two interfaces are provided:

  • User Interface (web pages): a series of on-line conversational dialogues consisting of web page interactions with end users that support the Gateway registration and enrolment process (see
  • Programmatic: APIs available over the Internet, comprising two types:
    • a secure web service interface. This is made available only to trusted users (e.g. government websites and applications), providing authentication and authorisation support, including the ability to register and enrol a user
    • a public web service interface with reduced privileges, available for any website  or application to conduct routine tasks including the authentication of users


Figure: API methods exposed

[Source: UK Government Gateway Frequently Asked Questions 2005]

A Simple Object Access Protocol (SOAP) interface is required for websites and applications to interact programmatically with the R&E API-based service. The parameters included in SOAP messages are required to be well formed XML documents conforming to the XML Schema (XSD) defined by the UK government. This enables maximum re-use across SOAP APIs and for both parties to correctly and consistently validate XML documents.


Figure: MoD use of the Chip and PIN support for online access to shared services

[Source: “The UK Government Gateway Remote Authentication. Jim Purves, E Delivery Team, Government Gateway. Department of Work and Pensions 24/10/2008]

PS. If you’ve got this far, well done. You might also find this previous blog useful in terms of some of the underlying architectural design: high level cross-government architecture — 2003 style

Posted in digital, future Britain, identity, IT, IT strategy, open government, privacy, public services, security, social exclusion, social inclusion, technology, technology policy | 1 Comment