The ghost in the machine

There's something weirdly detached about AI art. This is beyond the various "tells" that range from obvious to edge cases. It is this sense of there being no there there. A soulless-ness that is of a different quality than corporate art or mass-produced art or been-through-too-many-committee-meetings art. That's the backdrop that makes those glimpses of a real soul in there so striking.

The first level is seeing those human signs that are divorced from their original context. In Saberhagen's Berzerker series, the titular machines do not have vocoders or some other synthesized speech. Instead they talk using scraps recorded from the various humans who have been its prisoners. From word to word the age, accent, gender, and emotional content of the voice changes. And you get glimpses of those original personalities, the fearful, the resigned, the aloof intellectual, the child in terror.

It freaks me out as someone who has tried to learn visual arts, and who has studied the style and careers of various artists. Much of what the AI was trained on is effectively anonymous, as this was the output of commercial artists on contract or the various cogs in a studio system, or just artists you haven't yourself learned to recognize. 

But you still see those things, those brush strokes, those choices in line, the way a mouth is drawn. Things that in context are part of a complete style and approach. That come out of a philosophy (quite possibly one that is ever-shifting, as an artist approaches different projects with different intents, and their career changes over time as well).

But you don't see the same thing across the image. One line is drawn one way. Another line, right by it, is drawn in some different way. Not completely different (the AI is more selective than that) and not usually pure (it always blends a little). So you probably won't see an Arthur Adams mouth or a Rob Liefeld foot, not perfectly preserved in amber. But you will have a blurry glimpse, a funhouse mirror version, enough to know without the slightest doubt that this particular artist was in the training material.

And this lack of a clear overall purpose is also visible in something else. Something that is most marked when some semblance of it is present.

And that is in composition. On the largest scale, on the scale of the total image, the AI will probably have a harmonious composition. Most of the material it was trained on had that, and enough of it was using similar rules (or, more precisely, a small set of different compositional approaches that themselves have a small set of fairly well-defined rules). 

And some of this is because the current most popular approach of generative AI for visual art is progressive approximation, working from the broadest outlines in. So you could say the AI starts with blocking, with a massing study, and that is probably decent. Where it falls down is because there isn't a guiding purpose that makes sure that everything else unfolds according to it.

That's why AI is currently so shit at lighting, by the way. And yes, the AI-boosters will point out that it is "getting better" and will continue to get better. But it is doing so by brute force and band-aids. When the Hildebrandt's painted, they had a clear vision of sunlight (usually) coming from one direction, the reflection of what was often green earth coming from another. This was in their heads with every shadow, every shading of every limb or rock or tree.

And it was in the planning. When an artist like Jack Kirby penciled, he knew where there would be blocks of solid black, and he planned and placed them as part of the composition. They didn't just fall where you'd expect a shadow to fall.

The AI can capture this statistically. It places the shadows where they usually fall. It shades because the training examples shade. It only gets it right most of the time because most artists are doing the thing in similar ways and with enough samples it will probably not be led into producing an outlier.

But that's all aside of where I was going with this. There are AI images which go beyond simplistic harmony in their composition and actually move on to telling a story within it. Where there is a focus and eye-leading, where the pose isn't just a collection of average limb positions but actually communicates intent. When a character is engaged in a purposeful action that can be read.

And the reason is because there was an original image. I recently watched a video about a young artist (who has learned better since) who did quite nice drawings then sent them into AI to make them look more polished. His style was perfect for this, by the way. AI gets confused by linework and isn't going to save it anyhow, but blocks of color tells it the ideas you want it to flesh out.

There are other routes to this soul transplant. One is by over-prompting the AI to copy a single original artwork you've been inspired by. You might ask it for an astronaut on the moon and with any luck at all you will get one of the three top images from Apollo 11 that appear over and over across media. With a few screwy details, because it is still mashing together multiple sources and not all of them came from the photograph in question or even from the real space program. Or you might outright name the artwork and/or artist, and surprise surprise, you can absolutely get the Mona Lisa back. 

But there is also image to image. And that is, in the essentials, using AI as a Photoshop filter on an existing artwork. Or photograph, or whatever.

In fact, there are entire models that are absolutely and unapologetically based on this.

In ComfyUI, one workflow -- right there on the front panel of the application -- is to take an image of a person (which could be the actual original photograph of a real person) and a screen-grab from a movie or a TikTok, and put the choreography of the one on the other. This isn't an edge case or an abuse of the model, this is how it is intended to be used. ComfyUI supports plenty of models which are text-based and even in the above the final results can be shaped by a more typically generative process, but this is absolutely making a paste-up of two real (stolen) things. It is necessarily so; the toolchain doesn't work if you don't provide these two pre-existing things for it to mash together.

What is sad here, though, is the glimpses of a potential partner. There are parts of making art where the amount of expression you put in is too low a ratio for the effort put in. Inking is wonderful stuff. Erasing the pencils, not so much. And few people want to grind their own pigments.

Somewhere in there is the ability to partner, to have a dialog, to use technology to draw the inbetweens or fill in the blocks. Some of it, we have. Some of that has been with us for a while. There's a reason why Photoshop survives the shitty business practices of Adobe. 

Just, this thing we currently have that we call AI is being applied to the wrong parts of the process, and too often for the wrong reasons. We could have a tool that would allow us to work as artists, and automate the parts of the process that are least human. Instead we have a tool we are using to bypass the parts of the process that are the most expressive of artistic intent; to do those things that should be human.

(And, in the end, fuck up the small details we might have turned to it to do.)

Also... have you looked at the price of RAM lately?

Turducken

I don't know why I'm rushing to find another project. I'm finally into the meaty scenes of this novel. The stuff I was looking forward to doing when I started this book.

Took three days to hammer out a draft of the "Footloose" scene. That's one of those scenes that's not in the outline but comes organically out of the story. From the first moment I introduced Penny to Alamogordo and sent her to a Blake's Lotaburger, I realized there was a thing I could do there. 

It grew, until it became a scene where Penny confronts a bully like something out of an '80s movie -- a connection she, with her Media Arts degree, makes herself. Which is why the scene indirectly references Flashdance, Back to the Future, Footloose, and War Games. But I also name-drop the Marianas Trench and Manchester United. 

(You might also count in Breaking Bad, as a Blake's makes frequent appearances in that as well.)

It ain't about the name-dropping. That's just an observation. The scene is about how we see ourselves in movies, how some of the plots in movies reflect unhealthy trends in our society, and the point of it is Penny finding a way not to play out those tired old stereotypes. But it is really a side-note scene, at best the resolution of a tiny sub-plot; the real thing going on is pulling her off the path of solving the mystery no matter what, and placing her emotionally where she can take a different path at WIPP as well.

Anyhow, I've been thinking less of potpourri, and more of sequences that stack multiple elements to make something bigger than one alone. And I've been trying to figure out a simple way to describe one of these Turducken set-pieces in the Horizon Zero Dawn series, that becomes one of the more memorable sequences in the second game.

Our protagonist, Aloy, travels to the ruins of Las Vegas, now half-buried in the sands, on her quest to rebuild the terraforming system needed to bring the Earth back from disaster.

When the terraforming system was attacked, key sub-functions achieved a sort of unhappy self-awareness and fled to whatever distant surviving servers they could use as hosts. POSEIDON took refuge in Las Vegas, its arrival triggering the old desalinization plant and flooding the ruins of a grand casino that was once the center and showpiece of Vegas -- a Vegas already rescued from the desert once, through the efforts of a man named Stanley Chen, both investor and inventor of that desalinization system.

With me so far? A lot of stories might have stopped at the vista of ruined casinos overtaken by the desert sands. That's pretty spectacular already. But a big part of the adventure takes place in the transformed lower floors, now filled with water and the holographic illusions of sea life as a drowned god dreams.

In the middle of this mix is a trio of Oseram delvers, their leader driven by visions of the Vegas that once was discovered then lost again by his father. Delvers who are also showmen and who make the choice in the end to stay and to rebuild the town once again. And all three have amusing quirks and work well off each other; these are hardly throw-away side characters.

And as you the player explore, completing the challenges and puzzles to return POSEIDON to the task of saving the world, you encounter recordings that outline the story of Stanley Chen; his betrayal by his own business partners, his long-shot gamble at reviving Vegas, his success, and his final lonely trip through the town he had saved and made his home as the terraforming fails and the deserts sweep in again...leaving the computer core running out of sheer nostalgia, never dreaming of how that would one day save Vegas again in ways he could not have imagined.

Of course, all of this is set against the story of the Horizon Zero Dawn series, and Aloy's own personal journey. Themes and plot and story and character are all near-seamlessly interwoven with game mechanics. The very traversal mechanism (a sort of magitech SCUBA mask) that you use comes from those Oseram delvers and their personal story arc. (One reviewer used the subtitle "Gear and Clothing in Las Vegas.")

I love it when you can pull together different elements like this. I was recently trying to talk about this in a Reddit answer to a writer's question about using AI for inspiration. I am split here. Stated baldly, you could assemble one of the combinations mad-lib style. Or with a dartboard, or via ChatGPT. The tough part is joining them in writing.

But I think it is more likely you would come up with a winning combination because you understood the kinds of connections that worked for the story you wanted to tell. So you aren't trying to force something from a limited selection into working. Instead you are open to ideas, so when you are in the middle of constructing a story or story part, you recognize the germ of a thing that could be added in as it floats by in the form of a random news story, a face in the crowd, a spilled cup of coffee. Instead of an external generator of ideas, the story itself generates them out of wisps and whispers.

I do still expect a few more of those. I recently added a sort of apocalypse log to the Atlas Missile sequence, which I hope will make that more interesting (and is also my current solution for getting both Project Pluto and the next clue for the mystery into what otherwise is a bare silo).

The beats weren't quite working. This would be a good time for Penny to make some wrong choices, to fail a little, and for once not be so analytical about things. 

So Wargames is out, but Kubrick is in. Not the one you are thinking of, either. This isn't a line quote but a cinematographic quote; the Kubrick Stare.

Beta reader

ProWritingAid has really been pushing their AI beta reader. Many, many emails and popups and so forth on the sale of the credits necessary to run the thing. With the subscription I already have comes a small number of "free" credits so before I succumbed to the "now 35% off the 50% off of the special sale price!" Black Friday/Cyber Monday stuff I should really, really try the thing out.

I did.

It is AI.

Okay, it took a while to find how to activate it at all. All of the buttons went to the sale page, not to the "run this thing on my existing credits" page. And full compatibility with Scrivener requires putting ProWritingAid in as the always-on grammar checker and wanna-be Clippy for every single bit of text you type on your Mac. Not something I wanted.

But I was able to finally find it on the web version of the software and threw an opening chapter at it.

On the positive side, it seemed to grasp what it was reading. And possibly answered the biggest question I have about my writing (probably, that every writer has); does the reader understand what I'm saying?

Possibly, because this is AI. Which is to say, all it knows is that my text has the same text-shaped objects that it has seen in the other text-shaped objects it has been shown. Which might have been examples of good writing from good writers, or fanfic dredged from wherever it could get it.

The fact that it seemed to understand the three character names as belonging to, well, characters, is a trick that ELIZA was capable of. And that program can be emulated in a few dozen lines of BASIC.

On the downside, it praised the sample for having a fun and engaging narrative voice, and for weaving the modern-day setting with historical information. But, shit, that's what I was trying to do. So I made text-shaped objects that my meat-brain thought looked like the text-shaped objects made by writers who could actually pull those things off. I borrowed ways of saying things that I'd seen other writers use. 

So the silicon-brain agreeing that I'd accomplished my goals is really it saying, yes; I'd borrowed things from other writers it had seen. What does it bring to the table that allows it to tell if I pulled it off? In what way does it replicate the experience of a human beta reader?

(The pic above is of course the human, Beta; clone-sister to Aloy of the Horizon Zero Dawn franchise.)

The effusiveness of the critique gave it the flavor of friends-and-family feedback. That is, praise you can't trust, especially as it is so content-empty. As with all things, the most trustworthy things the AI spat out were the few small criticisms it was willing to risk.

Yes, I am very suspicious. LLMs are being trained both evolutionarily and programmatically to coddle the users. An AI that criticizes and corrects is going to be less popular and in the end sell fewer copies. And from all the sales, ProWritingAid really wants to sell some copies. So a tool from them that praises my writing is a tool I can not trust.

Even that, I could work from. Except for the so-very-typical empty AI phrasing. The "many writers have agreed that this may be a better way to phrase..." stuff that ends up saying almost nothing, but wrapped in language that does its best to hide the lack of anything inside.

The other tools of ProWritingAid are more useful. Sure, it is wrong a significant amount of the time, but it is absolutely clear about which words it doesn't like and why it thinks those words are wrong. So you can work with it, looking into everything it flags and checking, yourself, to see if it found something that should be corrected.

The same sample I threw at the AI engine was automatically sent through the checks for spelling and passive voice and so on. And it found things I would fix. But on the gripping hand...that chapter had already been through ProWritingAid. A couple of years back, but...it missed the stuff then. So what is it missing today?