People have been writing cross-overs since, well, before writing. There's something about having your favorite fictional characters meet that has interested story tellers all the way back to the oral tradition.
Fan fiction is huge these days. The internet is a lot cheaper than mimeograph stencils, and reaches a potentially wider audience. And fan fiction is by its very nature subversive to the original body of work: even if it intends no more than to add another story, some "continuing adventures," to those that already exist. The Seventh Doctor finally gets some decent stories, the Enterprise gets another five year mission with the original crew, or we find out just what Lando was doing with the Millennium Falcon before he unwisely played cards with Han Solo.
All the above examples are from fully licensed productions; books, animations, comic books, radio plays. And just by existing, just by telling one more story, they add to and they subtly change what had existed before. The James T. Kirk of New Voyages inevitably becomes different from the character that William Shatner portrayed: even the Falstaff of Merry Wives can not quite be the Falstaff of Henry IV, and that was from the same author!
Fan-written materials are much less constrained to stay within the official canon. They are free to do such things as bring other voices and other viewpoints into a universe. LGBT voices, for instance, in ways that commercial broadcasting is still not quite open to. They are free to go in various directions more suited to the tastes of their authors; darker and grittier or (more rarely, I suspect!) lighter and fluffier. More sexually charged. Including more historical elements. More militaristic. Whatever the particular interest, expertise, or fetish the individual writer (and hopefully a receptive audience) has.
Fan fiction also joins the existing body of alternate history fiction in the asking of that ever-popular question, "What if?" A question attractive enough that many mainstream works have included it; the Mirror Universe episodes in all of the Star Trek series, for instance. Incautious wish spells in Buffy the Vampire Slayer. What If stories in the Marvel Comics line. "Turn Left" in New Who.
This is something that, to me, gets at the heart of what writing is; the working out of implications, whether it is a premise in an original science fiction story, or a historical Want Of A Nail. And of course choosing which way to go with the potential implications, finding those that have the most useful dramatic potential, and figuring out how to properly dramatize them. Particularly in a series with often existential stakes like Doctor Who, the most logical and likely result of any changes is "everybody dies." But that lacks a certain something as a decent story!
Which brings us at last to cross-overs. There are many kinds of cross-over, with many different reasons for them. For instance, there is the tradition of the pastiche; writing works purportedly in one tradition, but using the voice of a quite different one. Sherlock Holmes stories written as if by A.A. Milne*, for instance. Then there are stories which transplant one cast (or as little as one individual cast member) from one canon to another, integrating them into that world as if native born (as in, the brilliant naval commander Captain Holmes, with his medical officer Doctor Watson.)*
What I meant to muse upon at this moment, however, are those stories where each canon has full and independent existence, and characters and situations from each intrude into the other.
Roughly speaking, these happen most around either works that are arguably compatible and contemporary (there is no reason, for instance, why Jim Rockford, Thomas Magnum, and the Simon brothers can't exist in the same universe), or when one of the fictional universes includes a facilitating mechanism; the Stargate of, well, Stargate, or spells in Charmed, or of course any random madman in a blue box with a good timey whimey excuse for being where the action is.
In the former case, many 70's and 80's television shows actually did connect, with even such oddities as Magnum showing up on Murder, She Wrote. As do, surprisingly, several of the properties of the newspaper comics world (Lois, of Hi and Lois, is Beetle Bailey's sister). And of course you have such one-publisher connected monstrosities as the Marvel and DC universes, in which all of their separate titles are wadded together into one complicated multiverse.
The tendency in fanfic is to lean towards similar aspects...science fictional worlds find each other through worm holes or transporter errors, vampires and witches hang out despite certain canonical differences, sword-and-sorcery fantasies tend to visit similar. Connected somewhere here is the "vs" sub-genre, which could be understood as narrowly as the question of whether the Enterprise could outshoot a Star Destroyer, but in my mind is pretty much the same game as asking who is stronger, Galactus or Superman.
One peculiarity of these mash-ups that occurs surprisingly often is when one body of work is explicitly fictional within the other. If Spider-Man swung by Joey's apartment in Friends*, they would presumably recognize him not as a known Manhattan hero, but as a fictional character from the comic books. Explaining this paradox is left as an exercise for the writer.
But this opens up into what I find is a prominent defining element to most cross-overs; and that is the question of who plays "host."
Works of fiction are complete. They contain all the elements necessary to tell a story (that is, the specific story they are telling). If there is a central conflict, it is necessarily balanced; if all of the vampires switched to uzis* (Darla was on the right track there!) the Buffy series would be much shorter. Which means two things. One is that there isn't a lot of space left to introduce new elements and complications. In fact, anything that made it into multiple seasons as a television series has way too much kudzu growing in its lawn already. The other is that anything a cross-over brings in is going to be unbalancing.
Okay, so this is part of the fun. Drop the Punisher into Inspector Gadget* and you've got a curb-stomp battle. Send Moon Base Alpha into the Empire,* and you needn't even bother to lift off all Eagles; it is going to be Bambi vs Godzilla.
But as fun as this is, it also makes for a short story without a lot of interesting conflict. But what to do? You can try playing "A Private Little War"; don't just import a lost-through-the-wormhole Defiant with full crew to support the local heroes, but follow them up with a Borg Cube. But if you sent both of these to, say, Sunnydale*, then the Slayer might as well stay home. A couple of teenagers with pointed sticks aren't going to make any difference to the coming battle. Which means for all intents and purposes, you've just written a DS-9 story.
The idea of the "host" universe, as I construct it, is that one canon sets the ground rules. Not just how the universe works, but also what kinds of stories are going to be told. If, in the above, Deep Space 9 is the host, then our focus is on Worf, Sisco, Quark, or whomever your favorite characters are, finding themselves on 20th-century Earth with things going on that they never read about in their history books. It is, in short, a planet adventure for them, and probably plays out with the usual combination of landing party getting in trouble whilst above them, the ship is threatened. As a Trek adventure, it is more than likely they'd conclude with the vampires stomped out and the Hellmouth closed; basically, the Buffy series would end there.
If, on the other hand, the Buffyverse hosts, then the outsiders would be unprepared to deal with vampires and demons and all the other things that go bump in Sunnydale's night. They would at the very least require the assistance of the Scooby gang. It might end up in the same place in the end, but it is certain Buffy would have a lot more to do then stand back and watch the phaser beams fly. Like the Borg, the host story assimilates all newcomers.
Some of this "host" idea is about the cosmology of each of the properties in question. Some properties have a more ready explanation that outsiders could be integrated into. The Buffverse, for instance, implicitly recognizes both alternate universes (and travel to such), and the ability of magic to make fictional realities real. Star Trek, on the other hand, would have to reach for other tools. If Darth Vader shows up in Star Trek, he has to have travelled from a galaxy far, far away. And if Alice shows up with White Rabbit in tow, you are going to have to blame the "Shore Leave" planet. Or the nearest Q.
Of course, this entire exercise is vastly easier for all of those canon that implicitly include a masquerade. Stargate SG-1 plays perfectly well with any other late 20th century property, from West Wing* to NCIS* -- up to the moment Ha'taks show up over Washington, that is! Primeval gets along perfectly fine with anything from Torchwood to Harry Potter -- at least until there's a herd of apatosaurs booking down the I-90. (And even then, one suspects the wizards wouldn't notice.)
Which in a way comes right back to that central problem. Cross-overs tend to break the properties that cross over. But then, isn't that implicitly part of the game? For want of a nail... At one level, yes, a cross-over is about Katherine Janeway and Dylan Hunt comparing notes about working with an unruly mixed crew of reluctant volunteers.* Or Connor MacCleod getting a chance for a friendly sparring match with Usagi Yojimbo.* Or the "2013" Lara Croft finding a Sam (Carter) who is really worth her* (I can not believe that story hasn't been written already.)
But at another level, it is a story of when things change. Usually radically.
(Thoughts partially inspired by discovering a whole slew of "Halloween" fics set in the Bufferverse. Here's the sich; in the original series, butt-monkey Xander dressed up as a soldier one Halloween, and due to the usual Green Rocks -- err, to magic/Hellmouth/etc -- ended up gaining lot of military skills plus -- at least for one night -- a working M16. Hence the fics in which he dresses up in such variety as Obi-wan Kenobi, a Super Saiyan, or Superman. Which of course rapidly break the series bible in all ways large to small, from the basic overarching plot to all of the internal character dynamics.)
(And, to my mind, not in a terribly interesting way. Part of the raison d'etre of the series is female empowerment, reversing the trope of, as Joss Whedon put it, the little blond girl attacked by monsters and in need of rescue. Giving any of the male cast this much power shifts that focus fatally. But even giving Buffy herself more power unbalances the central lessons of loss and sacrifice. Hers is an unfair universe, in which you win only a day at a time. I have yet to see a "Halloween" fic that makes Buffy's choices meaningful. I suppose, then, that is a challenge!*)
*All plot bunnies are released into the wild for whomever wishes to care for them.