Beta reader

ProWritingAid has really been pushing their AI beta reader. Many, many emails and popups and so forth on the sale of the credits necessary to run the thing. With the subscription I already have comes a small number of "free" credits so before I succumbed to the "now 35% off the 50% off of the special sale price!" Black Friday/Cyber Monday stuff I should really, really try the thing out.

I did.

It is AI.

Okay, it took a while to find how to activate it at all. All of the buttons went to the sale page, not to the "run this thing on my existing credits" page. And full compatibility with Scrivener requires putting ProWritingAid in as the always-on grammar checker and wanna-be Clippy for every single bit of text you type on your Mac. Not something I wanted.

But I was able to finally find it on the web version of the software and threw an opening chapter at it.

On the positive side, it seemed to grasp what it was reading. And possibly answered the biggest question I have about my writing (probably, that every writer has); does the reader understand what I'm saying?

Possibly, because this is AI. Which is to say, all it knows is that my text has the same text-shaped objects that it has seen in the other text-shaped objects it has been shown. Which might have been examples of good writing from good writers, or fanfic dredged from wherever it could get it.

The fact that it seemed to understand the three character names as belonging to, well, characters, is a trick that ELIZA was capable of. And that program can be emulated in a few dozen lines of BASIC.

On the downside, it praised the sample for having a fun and engaging narrative voice, and for weaving the modern-day setting with historical information. But, shit, that's what I was trying to do. So I made text-shaped objects that my meat-brain thought looked like the text-shaped objects made by writers who could actually pull those things off. I borrowed ways of saying things that I'd seen other writers use. 

So the silicon-brain agreeing that I'd accomplished my goals is really it saying, yes; I'd borrowed things from other writers it had seen. What does it bring to the table that allows it to tell if I pulled it off? In what way does it replicate the experience of a human beta reader?

(The pic above is of course the human, Beta; clone-sister to Aloy of the Horizon Zero Dawn franchise.)

The effusiveness of the critique gave it the flavor of friends-and-family feedback. That is, praise you can't trust, especially as it is so content-empty. As with all things, the most trustworthy things the AI spat out were the few small criticisms it was willing to risk.

Yes, I am very suspicious. LLMs are being trained both evolutionarily and programmatically to coddle the users. An AI that criticizes and corrects is going to be less popular and in the end sell fewer copies. And from all the sales, ProWritingAid really wants to sell some copies. So a tool from them that praises my writing is a tool I can not trust.

Even that, I could work from. Except for the so-very-typical empty AI phrasing. The "many writers have agreed that this may be a better way to phrase..." stuff that ends up saying almost nothing, but wrapped in language that does its best to hide the lack of anything inside.

The other tools of ProWritingAid are more useful. Sure, it is wrong a significant amount of the time, but it is absolutely clear about which words it doesn't like and why it thinks those words are wrong. So you can work with it, looking into everything it flags and checking, yourself, to see if it found something that should be corrected.

The same sample I threw at the AI engine was automatically sent through the checks for spelling and passive voice and so on. And it found things I would fix. But on the gripping hand...that chapter had already been through ProWritingAid. A couple of years back, but...it missed the stuff then. So what is it missing today?