How does one compose music for a play? This essay will likely be short and rambling; I know only a little of music theory, music history, and ethnomusicology, and although I have composed for many plays, the number of shows I've composed complete original scores for is very small.
Generalized work method first. I watch rehearsals, listen to the actors to get a sense of the pace and timbre of their voices, look at costume plots and set renderings to get an idea of color, complexity, and so forth. And talk to the director. A lot. If the show is a period piece, I believe in immersion; listening to music of the period day in and day out, until I start to think a little in those patterns and styles.
As a general rule, slavish imitation of a style or period is out. First off, it may not fit the needs of the play as well that way. Secondly, you probably don't have the skills to carry it off. So you are better off doing something that carries the feeling of a musical period or idiom but is otherwise written as music for the theater (and the needs of that specific production).
Compositional tools: Melodic is attractive. A full melody is the most powerful way of linking an idea in the play to a specific musical shape; a full melody reaches deepest into the emotions of the listener. The downside to melodic writing is it forces you to write in full measures. This can prove difficult if you only have eight seconds to cover a scene change, or if you have to trim or increase the length of a composed piece by an odd fraction of a measure when the timing of the scene changes in rehearsal.
Motivic writing sacrifices some of the power of full melody in return for much greater flexibility. A motive can be stated in a very brief time. Motives can also be inverted, convoluted, sequenced, contrasted; they can develop in fresh musical ways while still retaining their recognizable character.
The great strength of motivic writing, besides the ability to write to any length, any tempo, and any thickness of texture, is that you can develop them; show the same motive in heroic light, as a comedic stumble, in a sad minor key, etc. The downside, besides the lessened emotional impact compared to a full melody, is that fewer of the audience will recognize the motive implicitly (as they will a melody). For many in the audience, the motive will simply read to them as "some music is playing."
Of course a motive can be extracted from a melody, or a melody constructed from a motive. This is the best of both worlds. When I write motives, I try to say something about a character; small and every-retreating steps for a shy character, bombastic leaps and little flourishes for a Cyrano, a sighing motif for a love interest.
Loops. I'm not speaking here of store-bought loops, like Garage Band packages or Fruity Loops, I mean fully-composed chunks of music that are designed to repeat. Both motivic and melodic writing allows loops; the former are shorter and more controllable, however. Most music software can handle multiple loops of sequenced material. Some can even apply such things as changes of instrumentation or changes of key. Thinking in loops isn't really that different from thinking in motives or melody; it is mostly thinking about music in terms of interchangeable parts that can be switched, duplicated, etc. as needed to fit the time given for the finished piece of music.
(A friend of mine with many more theatrical compositions under his belt uses loops like this, and I see the results in his hands. My tendency is to develop, instead, and to use bridges and repeats to fill those extra spaces that turn up during the rehearsal process).
Rhythmic writing. In this method of attack, you establish a beat, and the musical material on top of it is secondary. I wrote most of Agamemnon this way, precisely to avoid the problems of cutting music too tightly to action. Since nothing went outside the framework of a one-bar figure, I could fade out at any arbitrary point. Arpeggios, pedal points, ostinati are the classical tools here; drum and bass are the modern ones. Pads are more-or-less arrhythmic ways to provide a texture.
Textural. In this method, you eschew traditional melodic development, and even classical harmony, and concentrate on the pure sound. With classical instrumentation, it helps to write in a whole-tone scale; that way, no note is ever a "leading" tone, and no cadence ever resolves. Instead the music just goes on, a tapestry of shifting textures. Textural is of course equally effective using synthesizer pads, found sounds, and the like.
One interesting advantage to the more open forms, like rhythmic or textural writing, is that you can write to cross-fade or even layer on other material as the action of the play evolves. Against a softly murmuring flow of woodwinds, a spiky clarinet cam signal the appearance of Puck; and a thickening of the ongoing woodwind textures with strings the effects of the love potion.
Software is making possible the triggering of moments like this so they fall in tempo. As of the last show I used this idea in, however, I had to use my own ears and sense of rhythm to hit the beat!
Already, there are software options where a key change could be executed by operator control in the midst of playback, or the instrumentation changed, or the tempo altered. A program called "OrchExtra" is already in use replacing pit orchestras (a mixed blessing indeed); OrchExtra allows a conductor the same control over tempo, including fermata and vamping, as they have with a live orchestra. These sorts of tools are increasingly within reach of the sound designer as well.
Writing for plays is like writing for Television and the Movies, and unlike writing for the Concert Hall. Ideas should be, as a rule, simple, clearly expressed, broadly emotional. Say Scene Three ended with Susan alone and crying at an empty table, thinking she'll never see Rick again, and Scene Four opens at their wedding. In the eight seconds it takes for the stage crew to push around the scenery and the chorus in their wedding outfits to get on stage, you've got to carry the audience through this emotional change, AND let them know it is Susan getting wed (as she's still in quick-change for the first five minutes of the new scene). You can't do a recapitulation, then a nice segue, then a full statement of Susan's theme; you haven't got the time. Sometimes you have to let subtlety fall by the wayside and get right to business with the starkly obvious -- by playing Wagner's wedding march, for instance! (Although you might chose to play it in a minor key, or destroy the closure of the cadence...)
When you are writing under dialog, it has to stay away from the voice. Percussive sounds like guitar and piano work well. Sustained, vocal sounds like flute and clarinet, not so much. When possible, stay out of the pitch range as well; 'cello below the voice, high violins above the voice, but nothing in the same pitch range as the voice.
In the movies, the music is finished and cut in after the dialog is recorded. They can get away with brass and drums in the middle of a speech; because they bring those in during the natural gaps in the speech. We can't do that. At least, not yet!
Which brings us to timing.
Assume you do want to "Mickey Mouse" a scene. My composer friend, for instance, got to score a sword fight as if it was an Errol Flynn movie. There are two ways to go here; the music follows the actor, or the actor follows the music.
Philosophically, I dislike the second. I prefer the actor to be able to adjust their performance to the moment. An audience's energy changes from night to night. An actor develops a scene through the run of the performance, finding new places to express something a little more deeply. Forcing them to hit the chalk marks at certain words or actions is, I think, a disservice to the production. Be that as it may; if you chose this route, build in musical moments for the actor to hear. Don't ask them to count bars. Ask them to make the cross when they hear the tuba.
For the former, lacking sophisticated playback methods, there are two methods which you can apply, singly or in combination. The first is keeping the timing loose. I made the tight changes in my Play it Again, Sam monologue happen near the start of the cue. The rest of the cue got progressively looser, ending on a vamp that could be faded arbitrarily when the actor finally reached the end of his speech. The second is building in "catch-up" points, which is what my friend did for his sword fight. He wrote in a long sustained note that would happen close to when the fighters were in a blade lock, then the operator would take the next cue, ending the fermata, when the lock broke.
Increasingly, instead of having to move from one CD cue to another, you can layer audio tracks from playback software. So in the example above, you could do the fermata as described...but you could also add an a-tempo glissando for the moment Barrymore swings on the chandelier. Depending on the skill of the sound operator and the sophistication of the design, it can sound as if it is just one complete and contiguous piece of music, not several different things being layered and spliced.
A tangential remark to the above. It used to be, Sound Operator was a technical job. You got someone with lots of skills and training in not just running the complex sound equipment, but in maintaining it. As technology has given us simpler, and usually more robust, playback mechanisms it has also taken away; sound operators in smaller theaters now tend to be under-trained volunteers. Or there is no operator at all; the Stage Manager will take sound cues with their free hand (while watching the stage, calling light cues, and following along in their book).
Oddly enough, the place you get the best opportunities for creative application of sound is in the musical; most musicals even in smaller venues amplify the actors (usually with wireless mics). This means a skilled operator mixing (and maintaining) those mics. With canned music ranging from CDs to OrchExtra (as happens in the more budget-conscious productions), you also have a trained musician available. This means that adding in sounds or musical material that require a comfort with technology and good musical sense is possible in those venues, where it may not be possible in a "straight" play.
Really, at this point in the game, if you want to do a Mickey-Moused, evolving, tightly-following musical design for a straight play, you either need to be designing at a major regional theater -- or you are going to be operating the sound board yourself.