Today, the future of the Union is a hot and controversial political topic. A century ago the outlook could not have been more different.

On the eve of the Great War, it must have seemed a rock of stability in an uncertain world. It was such a fact of life that no one of any influence questioned its future. If truth be told, the Scots had been remarkably fortunate. Rightly or wrongly, they assumed that their global economic eminence was rooted in the Union. But that wealth had not come at the expense of either cultural dependency or loss of identity. Whether they knew it or not, Scots were having their cake and eating it.

But between 1914 and the 1950s, this almost smug relationship was assailed to an extent unknown since the eighteenth century. Despite final victory, World War I was a human catastrophe on an enormous scale for Scotland. What began as national euphoria had degenerated by 1918 into dark pessimism. According to one Great War historian, Scottish regiments suffered more combat losses per capita than any other nation. Whether this verdict is reliable or not, it is clear the slaughter of the nation’s young men on such a scale was entirely unprecedented.

This catastrophe was followed by the collapse of the markets for Scottish heavy industry in the late 1920s, which contributed to a remarkably high level of emigration. For the first time since census records began, there was a fall in the Scottish population. Edwin Muir eloquently captured the crisis of national confidence in his Scottish Journey of 1935: "Scotland is gradually being emptied of its population, its spirit, its wealth, industry, art, intellect and innate character." Another contemporary commentator, George Malcolm Thomson, was even gloomier: "The first fact about the Scot is that he is a man eclipsed. The Scots are a dying race."

Yet despite all this, the Union remained impregnable. The Conservative and Unionist Party in Scotland was hugely popular between the wars, winning five of the seven general elections during that period. During the long economic crisis of these years, Scottish voters preferred the secure umbrella of the British state to any nationalist adventure. The foundation of the Scottish National Party in 1934 showed that not all Scots were in the unionist camp – but its successive failures at the polls demonstrated conclusively that the vast majority were.

The outbreak of the Second World War further strengthened British identity. For a time, plucky Britain and its empire stood alone against the massed forces of the Third Reich. Every nook and cranny of life was affected as the nation geared up for total war. The age-old distinction between combatants and non-combatants faded as the civilian population on the home front struggled against enemy bombers, food shortages and the fear of invasion. The legend of a Britain united behind a noble cause endured in the folk memory of the post-1945 generation through the endless popularity of war comics, books and films.

This was not the only vital factor buttressing “Britishness”. The foundation of the Welfare State in the late 1940s and the commitment to full employment in the post-war world had enormous appeal for Scots, who had suffered the full impact of market failure in the 1930s. Even the beginnings of the end of empire with the independence of India and Pakistan did not disturb the union connection in the short-run. A new bond had been formed.

As living standards finally started to improve in the 1950s and the years of austerity faded into the past, unionism in Scotland seemed unchallenged. Indeed, in 1950 Labour dropped its long-standing manifesto commitment to Scottish self-government and the SNP continued to stagnate in political irrelevance. 1955 saw the Unionists achieve just over half of the popular vote, a unique and remarkable achievement in Scottish electoral history.

But this political consensus did not mean that "Scottishness" had in any sense evaporated. On the contrary, the mass interest in the Scottish Covenant of 1949, which advocated a parliament in Edinburgh within the union and attracted nearly two million signatures, suggested that Scotland's sense of itself remained robust. Moreover, by the later 1950s all was not well with the Scottish economy. The long period of Britain's relative decline against international competitors, which lasted from the 1960s to the 1990s, had begun.

The balance between “Scottishness” and “Britishness” now shifted. The rise of the SNP, the new and pragmatic interest in devolution by Westminster and a fresh vitality in Scottish culture were all signs of the times. A key decade was the 1980s, with the imposition of hugely unpopular social and economic policies by the Thatcher governments – not least the Poll Tax. The Scots had not voted for Tory radicalism, and began to feel that they were now suffering from an electoral dictatorship. That experience put more steel into the Scottish electorate and their politicians. Any ambiguity about the relevance of a Scottish Parliament to the future of the nation quickly receded.

More than half a century on from the high noon of unionism in the 1950s, the issue now is whether the time-honoured connection between Scotland and England will survive for much longer in the new millennium. In 2004, around three-quarters of Scots felt "exclusively" or "mainly" Scottish, a significantly higher proportion than the equivalent measures in England and Wales. These "Scottish" loyalties are especially common among the younger generation.

This need not mean that political independence is inevitable. It may be yet another manifestation of the Union's historic capacity not only for flexibility but for giving full and easy scope for the Welsh, English and Scots to express their cultural and ethnic identities within a UK framework. Perhaps inevitably, however, most recent comment both in the media and among academic analysts has been about the reasons for the decline of "Britishness" over the last half century. The obvious check list might include the end of Empire and Britain's subsequent fall for a time to the status of second-rate power; the huge and increasing importance of Europe and the parallel decline in the authority of the British state; and the ebbing of respect for the institution of monarchy. Again, since the end of World War II and the collapse of the Soviet Union, there is the loss of a clear "other" which can help to sustain British national solidarity against a common foe.

However, whether all this means that a political divorce is likely in the short-term is less certain. Three hundred years of Union have resulted in multiple familial, personal, economic and cultural connections between the two nations. Scots have long migrated to England in large numbers. Less well known is the continuous movement in modern times from England to Scotland. Between 1841 and 1911 a quarter of a million English and Welsh men, women and children came north. According to the 2001 Census, over 400,000 English-born were resident in Scotland, by far the nation’s largest immigrant group.

Not so long ago, it was possible to speak with concern about the "Englishing" of Scotland. More common nowadays is the reference to the “Scottish Raj” in English politics, media and London's financial institutions. The story about the Midlands MP who asked why the Scots should need a parliament when “they are running ours” strikes an ever-stronger chord. The Scots have frequently been the butt of English jokes; but such minor irritations have never really prevented them achieving access to the highest positions in politics, business or academia south of the border.

The economic crises of the 1970s through to the 1980s, which undermined confidence in the British state, had disappeared by the early 1990s. Balance of payments problems, hyperinflation and trade union militancy have, for the last decade or so, gone from the UK. Since emerging from the recession of the early 1990s, Britain has thrived and most Scots have shared in the benefits. According to the International Monetary Fund, the growth of GDP per person in the UK was both stronger and less variable than that of other rich nations in the G7 over that period.

But these good times have not led to a final stability in the relationship. The devolution settlement is not entirely fit for purpose and has too many anomalies to provide a real consensus on both sides of the border. Among the most contentious issues is the notorious West Lothian question: the conundrum that while Scottish MPs can vote on English domestic issues in Westminster, English MPs have had no say on Scottish matters since the foundation of the Edinburgh Parliament. Indeed, one of the novel developments of the past few years has been the rise of English nationalism, in part fuelled by perceptions in the south of mendicant Scots being subsidised by the English through the unfair generosity of the Barnett formula – the per-capita mechanism used by the treasury to calculate some aspects of public expenditure in Northern Ireland, Scotland & Wales. Why is there no English parliament? Why do Scottish MPs vote on English issues? Why should Gordon Brown, a Scot from a Scottish constituency, be allowed to rule Britain as Prime Minister?

It was a profound historical irony that in the very month of the 300th anniversary of the Union, a minority SNP administration first took power in Edinburgh. The pundits produced a variety of reasons for the historic success of the nationalists: these ranged from the unpopularity of the Iraq war to the mid-term malaise of the Labour government. Whatever the fundamental factors, there can be little doubt that the SNP victory has ushered a new period of volatility in the Union, making speculation about its long-term future even more hazardous.

Still, research published by the Institute of Public Policy Research North in autumn 2007 suggested that the SNP triumph had done little to bring Scottish independence any closer. The report indicated that only three in ten Scots supported independence – and that figure had hardly moved from the levels which existed even before devolution. Indeed, before the Scottish elections in 2007, only 63 per cent of those who intended to vote SNP said they supported its core policy of independence.

During the course of the election campaign itself, the numbers favouring independence declined. In the event the electorate voted for unionist parties by a considerable majority. To a large extent, the SNP’s success came not from a collapse of Labour support, which fell only slightly, but from the failure of smaller parties like the Socialists and the Greens to make a real impact. What the research did reveal was a widespread desire in both Scotland and England for changes to be made in the devolution settlement. English voters want such anomalies as the Barnett Formula and the West Lothian Question to be addressed by Westminster while a majority of Scots wish for more powers for the Holyrood Parliament within the framework of the Union.

Foretelling what might happen next has become even more hazardous in recent months. The SNP minority administration has made a remarkable impact during its short period in office and demonstrated a clear capacity for effective governance. In addition, the good economic times have come abruptly to a halt with the UK slipping into recession. Perhaps most crucially of all, the possibility of a Conservative government after the next General Election looms ever larger – an event that would be a major boost to the Scottish independence movement, according to recent polls. There is, therefore, then only one certainty as we gaze into the future: the unpredictability of the Union issue in Scottish politics.