When I was a kid, one of my absolute favorite aspects of genres like Fantasy and Science-Fiction was the ability to delve into and explore imaginative worlds. There's something tediously mundane about being young that seems to cry out for more interesting locales - if only in the mind's eye. Video games provided an added layer to this escapism by allowing me free reign in my explorations.
For many years - especially during the heyday of 'multimedia', there seemed to be a strong emphasis on converting gaming from a more exploratory pursuit into something that's commonly referred to as a 'roller coaster'. By keeping the narrative on tight tracks (e.g., linear corridors leading to specific cutscenes) game designers would be better able to control the player's experience, infusing it with the desired emotional beats, pacing, and outcome.
In my experience, there's always been a very unsteady tug-of-war between authorial control of a game's unfolding, and satisfying the player's desire to have agency within the game world. It's a precarious balance between something relatively open and unfettered, such as DOOM or Quake, to a careful balance of scripted and open sequences such as is found in the Half-Life games, to the rigidly linear, obsessively scripted Call of Duty titles (here using first person shooters as an example genre). As to my preference, I'd always rather play a DOOM-style game over a heavily scripted title because that freedom to explore and chart my own course is why I play games rather than spend the same time wtching TV or reading books.
Not all 'restriction' on exploration is bad, mind you. I appreciate the smaller, more focused environments of Skyrim and Oblivion as compared to previous Elder Scrolls games, for example. I've still got plenty of room to meander without forcing me to walk endlessly through fields of nothing. Making extra room to play in is only as valuable as there are things to play with. Many early titles were designed by folks still in the process of learning, and had little respect for our available free time (I'm looking at you, Ultima IV).
Basilisk Games' Eschalon: Book II does a great job of recalling the heyday of exploration in classic RPGs. There are many places to walk off of the beaten path, and the rewards for doing so are satisfying.
This last concept of making exploration meaningful deserves special consideration: I find it depressing that many mostly linear titles throw in useless side quests or pointless extra side tunnels in some strange attempt to argue that you're not actually simply playing an 'interrupted movie'. The rewards for this 'exploration' are typically trivial or, in the worst cases, non-existent. How many Japanese RPGs have I played wherein a side tunnel led to a chest containing some item that I could get in town for a few coins? Instead of encouraging exploration, these designs simply reinforce the notion that good things only come when you're on the invisible tracks.
What would be wrong with placing the best rewards in the game in average side-quests1? I remember that the best equipment I ever snagged in Fallout II was obtained in a random encounter. What an amazing reward for trying to map out every inch of that massive, endless waste! And what better encouragement to savour the game slowly, exploring every nook-and-cranny? I'm frequently reminded of that encounter when I get tired of the endless useless 'loot' based action RPGs that were all the rage on the PS2.
Video games aren't movies - in many ways I'm sure that their creation is a much more complex and nuanced process. The best games are open virtual worlds and the trick of their design isn't so much creating a compelling narrative, as in films, but rather creating a world that's more enjoyable to be in than our own, at least for a little while. Folks can have their roller coasters, but I want to explore!
1I realize that in many JRPGs, the 'best' equipment is found in side quests that are typically very tedious and stacked towards the very end of the game. Though some of these are interesting, I've found that the majority of them are just padding for completionists, and are poorly integrated into the overall adventure.