Why not poet

Software continues. Today at work I installed and learned how to use Meshmixer, re-learned Fusion 360 and rebuilt a model that went missing between accounts, refreshed my memory of the Prusa slicer and printed the new mesh and tried it out on the factory floor.

Fusion keeps changing. Even their trainers are admitting that the other videos are now all wrong...not as if they will apologize, though, or make the slightest effort to distinguish between functions in the software and functions locked behind a paywall.

Got home and sticking Automatic1111 on the M2 to try to improve workflow there. PhotoShop is migrated there. Oh, yeah, and installed the Satisfactory Mod Manager on the PC last night, and tried out a mod that does spline roads (roads are a pain in Satisfactory, which is a pity, because trucks are an efficient way to move things, are unlocked at an earlier game stage, and look cool, too.

Oh, yeah, and I had an emissions system-related repair on my smartcar, but it won't pass smog because the computer hasn't finished calibrating. The book recommendation is to drive for 50-100 miles (which is about 50-100 times longer than my one-way commute). I may have to do that on the weekend! Yeah, that makes sense...to make sure you aren't producing extra CO2, you need to drive for two hours (producing lots of CO2...)

still calibrating...

I'm trying to let Mibl work and not get distracted re-doing cover art yet again. Especially as I'm still deep in Part two of four. I got into another detour, contemplating the role of religion in this particular story. My "Sangreal" moment still isn't working, two re-writes later, but that's not even half of it now.

Because the remainder of Part II, and the crux that comes between II and III, is all about the Gates of Hell.

Cover

Trying out a new cover artist. It's a bit premature as I don't have a page count yet. I thought it might inspire me to finish the damned thing.

Also tinkering with the old covers again. I have ideas about doing some light adjustment and AI upscale, especially on that horrible hand in the third cover. I bought my own mask, shot a new pose, now it is into the mysteries of ControlNet to turn the pose data into an AI-generated hand, then paint that in with the photograph of the real mask and into the original reworked stock model.

And if I have any CPU cycles left, I shot the introductory instruments for a video of my "Game of Throne" cover on "viking" instruments (bowed lyre, bodhran, penny whistle, fiddle...) Which was in the low light of the LED candle effects I built for The Birds against a stone wall I built from foam scraps from a day when there was nothing else happening in the workshop.

And 3D printed an iPhone tripod attachment when my regular camera couldn't handle the extremely low light. 

I have the software installed on the new computer but haven't gotten to that, either. In fact, I haven't even migrated PhotoShop yet (have to de-authorize it on the laptop first).

I haven't quite gotten the AI tools helping with the cover art and other projects. They have managed to spin off a few images that I want to find time to haul into PhotoShop to fix the stuff that AI gets badly (like, um...hands...)

So makes sense the most I accomplished this weekend (outside of writing) was going back to a Satisfactory save and cleaning up a few things in preparation for Patch #8 to wreck everything...

And, yeah, just as well all those graphics efforts are on the other computer. Otherwise I might be posting a bunch of them here.

Notre Dame du Travail

Brought the car in for service and they kept it for several days. During the walk to work got some thinking in, which the book needed, but no pages written (which the book needs even more).

I have a new scene. The epilogue was a little weak to start with. But I was poking around at Atlas Obscura and I found this place. It's the steampunk church. Inside, it looks like a cross between a small European church and the Chicago "L." Same exposed trusswork -- same architect -- as the sadly-demolished Les Halles marketplace.

It was built just following the 1900 exhibition for the laborers who built that exhibition. It is a working-man's church. There's a local church here, the Julia Morgan, which was done in the Craftsman style of local woods to feel comfortable to the farmers of the congregation, people who had raised a few barns themselves. This church is the same idea, but a space that steelworkers would find a familiar space.

So I'm finally getting my "chevaliers de sangreal" moment. She can sit in the pew in this fantastic little church and think about what Jonathan Huxley was actually trying to say, how under his little scavenger hunt was a real secret.

But I'm still on the fence about going into the catacombs. It's over done, that's what it is.

Brains

I had a lingering sinus thing that had me incredibly brain-fogged. The fog lifted just in time for me to renew some registrations that were gonna end up in hefty fees if I delayed longer. And put in some "look good" hours at work.

And it has continued lifting. Now at the tail of a quiet weekend I'm starting to grasp the threads of the Paris novel. I'm remembering things. Yes, Rodin did do a sculpture of Victor Hugo, even if it was his Balzac that proved so controversial. And he loved his dancers -- although that ballerina with the ribbon is a Degas. Like that's a surprise. Doesn't mean Hux can't reference it.

But the more I read about Rodin, the more I want to do another of what I call the Hux-cams. 

This is not a dual-time story. It is mostly Penny, in the present, puzzling out clues in this book from the early '20s. But the character Major Joanathan Huxley does appear, seen in brief glimpses through his memoirs.

Huxley's word appear in four ways in my book. His clues are presented the same way as they are in his book; enigmatic epigrams opening new chapters. Of course I have more chapters than I have clues, so I had to add a few random quotes from him as well, and it has been tough coming up with something witty enough to make it look worth quoting. He's no Oscar Wilde. More to the point, I'm no Oscar Wilde, and I'm the one who has to come up with the stuff.

Penny also describes and synopsizes, sometimes with fragmentary quotes. But there's one more form. And that is complete multi-paragraph excerpts. So far I've written three. There's a description of the butte of Montmartre, probably from a good decade before the story from some of the details in it. There's a brief scene from getting out at Gare d'Orsay to reaching the entrance pavilion of the Paris Exposition of 1900. And there's a scene where he meets up with friends outside le Bateau lavoir -- it doesn't get named in this scene, not even in English.

So a bit at Rodin's studio would be great fun. Huxley claims to have posed for one of his students, and he also might have a fling with one...I'm leaning towards the Finn, Hilda Flodin, as the tragic Camille Claudel is a bit of a live wire in that particular span of years. But there's also the Welsh Gwen John...

Thing is, that would take a bit of heavy reading. In the previous scenes I've managed to mostly escape names and details that could be too easily found wrong. This would be very much those historical people.

Okay, there is a brief mention by Huxley as a chapter heading; “…at which stalls I found a copy of Willy’s Claudine à l'école, which tantalizing rumor had was largely the fertile imagination of his young wife, Sidone.” (it introduces the chapter where Penny visits the bouquinistes along the Seine and gets Huxley's book stolen off her by a young traceur. )

(Colette is rightly famous by the time he writes the book, but like me, he is slipping back and forth in time and in this quote he is expressing what his younger self had understood.)

***

Meanwhile, the second computer, currently stealing the second monitor, is installing Dreambooth to go along with the ControlNet installation (and pre-trained models) I'm adding to my AUTOMATIC1111 install. I have yet to get to grips with any of those. The window for this implementation of the DreamBooth code is...daunting. 

I did manage to get it to sort of do one of those future bedroom things you see as the static image on YouTube 10-hour soundscape videos. It is tough to teach the AI sensible shapes, but it excels at deep, rich, complicated textures and lighting effects.

There's another whole essay there, but for another night. Right now my biggest ambition is to get ControlNet up on solving the hand problem so I can re-render that weird hand on my second cover. And then try to turn it loose on the model for the second cover. Well...turn it reined in, just letting it add a little more detail and texture. Depth mapping may help there, too.

If I try any more art I need to start experimenting with painting the kind of rough that it wants to see. I was able to do that once; had a text prompt spit out an amusing image of some future cop type but the gun was really wonky. So I went into Windows Paint3D (what a ghastly program) and dashed up a magic-marker shape that got a little closer to the right composition. Threw that back in for inpainting and in just two runs it spat out a nice detailed glowy-bits future ray pistol thing.

So that's the idea; compose traditionally, throw that in with img2img or depth mapping (there's also an "interpret a sketch" pre-trained model in ControlNet that sounds promising), and let the AI shape the prompt on the basis of that composition. This is very different from how you would normally do digital art.

There is at current an annoying space between what the AI does well -- all too well, really -- and what we humans do well. The AI is lie the Salon, willing to spend the endless hours of drudgery smoothing out every brush stroke for seamless blends and those subtle spray-painted lighting and volumetric effects that are endlessly fiddly to do. The humans are Impressionists, capturing shapes and ideas and compositions in rough strokes.

Thing is, you can't get the computer to see things the way you do. And you can't blend the way it does. So the space where you can try to paint over its mistakes is a very wide gap requiring way too much of the most boring kind of work to cross.

And am I glad the blow-up of AI postdates the setting of my Paris novel? Damned straight I am.

One step closer to the abbey

Got my dual-monitor, dual-computer rig mostly set up. I have more desk space now. I need a better keyboard for the Mac side, unless I invest in a keyboard/mouse switch, too. Probably will borrow the Mac bluetooth keyboard I have in my shop...when my car gets out of the shop it is in. 

Progress on Sometimes a Fox. Had to pause to work out the next set of clues, and I've pretty much got the locations picked out that the last set of clues will send her to. As the story progresses, Penny spends more and more time getting to know Paris outside of the landmarks (famous and not) in the book she is following.

Like la Defense, the fancy built-up area around the Defense arc, where she'll take a couple of classes from a pack of traceurs who like to practice there. And still get her butt kicked at parkour, but that's the nature of these stories; she's gaining competence, but she's never going to become an expert martial artist one '80s montage later.

Hux's clues lead from the Rodin's plaster mock-up of The Gates of Hell, which Huxley figured would be at Hotel Biron by the time he wrote up his memoirs (the sculptor hadn't even moved in there when Huxley was in Paris), to the Palais Garnier. This has become almost a three-fer clue; Penny agonizes over trying to solve it and goes off in all sorts of directions, from thinking about Monet's Water Lilies (which have a dedicated gallery now at the Orangerie) to the Van Gogh exhibit; the digital one with the animated paintings. Which I caught in San Francisco but, yes, the original show was in Paris during the time I've set the novel.

And Nathan and his team go to the Van Gogh, too, so I can have her sneaking up on them under Starry Nights and through sunflower fields, but they've jumped ahead because the next clue is so obvious it could only be set at the Paris Opera House.

So she goes from the Pompidou center, where there is an exhibition of the en l'an 2000 "Steampunk" postcard set of 1900 whimsey, and the Herge exhibit, which was actually in Paris but at a different museum that year but never mind; I saw it at the Pompidou. And runs into parkour people playing on the outside of the building and that's where she gets an invite to la Defense.

Then makes the same leap Nathan did to Van Gogh's sunflowers, is lucky enough to trail him and his "rival team," -- because having another team of treasure hunters is so totally part of the pattern -- and finds out when they are going to the Opera.

And has a chat with a student who is amiably picking padlocks off the Pont de Arts bridge (I think that's the one that's currently illegal to put love locks on) and is reminded of her own efforts to learn that skill. Which sets that up as a Checkov's Skill, as TVTropes calls it, but it is false; Nathan's team have jimmied the door.

For no good reason they've worked their way down to the "lake" (yeah, not really a romantic lake, either) and are singing snatches of Phantom but it doesn't take being a Sondheim fan for Penny to be getting really pissed at them. So this is the II/III turn-around, when she decides she's not about friendly competition anymore, but is going to take them down -- or at least, get to the secret before they can cause any more damage.

Part III is largely the Bohemians, and the steampunk superhero comic book Bastien is drawing, and Amelia coming out as a wrench wench. The next clue was les Halles, which is less there, these days (they just opened a renewal version but the old buildings are gone). There's a clue there about singing stomachs. Then a clue to La Petite Ceinture, which is somewhat there although quite abandoned. There's access to the catacombs from there and cataphiles walking through, as well as all sorts of other things. Penny has an encounter with some toughs there, and this happens more or less as Hux finally gets around to explaining the time he and an old French veteran got jumped by some Apaches near Montmartre.

I think she meets up with her cataphiles (or whatever flavor of urban explorer) at the false building that's...in my other notes but it was convenient to the "little belt railway." I found one that's got a video of some guy breaking in to look at it.

And then to this park I've forgotten the name up, which was a fantasy excess built in the 1900s with artificial lake and a lovely grotto made out of the old Plaster of Paris mining operations. With a stop-over at that one station (pont de arts perhaps?) that got the full Jules Verne treatment, because this is the full-costume Steampunk Garden Party.

Where Bastien and Girard are shamed by the others into taking the love triangle with Amelia into a mock duel...although at this point Huxley has gotten to where the friend and roommate of his "excitable little bullfighter" -- one of the colony of Catalan artists atop the butte -- shot at his girlfriend then shot himself.

The reader may or may not have picked up that this guy is a young Picasso. He never gets named in the text. That's the pattern for this one. In Japan, Penny realized she was missing things. Here, she notices Huxley doesn't like Sacre-Couer, his clue is talking about the blood of martyrs spilled there, and right after that she's at the parc louis-michel. And doesn't get any of it. (Eventually she does learn more about the Paris Commune. And, yes, it does figure in the mystery).

The clue at the park probably points to Notre-Dame because I've no more I need to do with Huxley's stuff. There might be another location or two before I get there. In fact, probably will be. Including possibly a Parisian hackerspace. Plus her Bohemians do a special version of their cabaret show which recreates a famous kiss between Colette and Missy (in 1907). Dressed as archaeologist and mummy. Oh, did I mention Penny also finally gets to the Louvre? Egyptian collection, mostly, but gets enough afield in Mesopotamia to learn about Jane Dieulafoy.

Which means I need to write at least two clues before I can complete Part II. The one at the Gates of Hell, and the one at the Paris Opera House.

As far as I've gotten on Gates of Hell is something about the lillies that toil and spin. And probably something about fine raiment because that would also point to the same verse being paraphrase (King James translation).

And the Palais Garnier one is something about looking down from a private box. But nothing about a chandelier. Unfortunately I can't really work a lake into it; that Parc des Buttes Chaumont is a railway and a public market later. (Yeah, I stopped and looked it up). 

I thought having two monitors -- both of them now running at full resolution, too -- would give me enough virtual paper to spread all my notes out and solve this. Perhaps. But I just finished installing ControlNet into my Stable Diffusion folder and now I want to see if it can really fix those mutated hands...

Into the rabbit hole

Or should that be, "Digital painting of highly realistic brown rabbit sitting on grass, vivid lighting, bokeh, trending on Artstation."

Text-to-image is both less what someone might think, but has more going on under the hood than some might expect. Blinding entering prompts might get results, but even with this stage of creating AI art there is more of a creative involvement than that.

You could take a line, put "walked into an art gallery" at one end, work your way up through "hired an artist and worked with them through several drafts," to "did the pencils and handed it off to inker and colorist" and "limited myself to three random colors pulled from a box of pastels" along the line. But you can extend way past "painted to from scratch" to "ground the pigments personally" to... "invented a new art form?"

At all stages other things are intersecting. A random defect in the paper that looks like a cat curled up and inspires the artist, the art store was out of blue, a friend made a comment. At all stages in traditional art, one is using what one learned in classes, taking tricks from other artists, opening reference books on anatomy and using reference images.

At all stages the artist is engaged, using their eye and their skill to intelligently react to and to shape what they do in response.

AI changes things in...weird ways. It is like hiring an artist who you can only contact briefly through garbled emails, who you suspect doesn't speak your language, who doesn't tell you what they are thinking until their (often unexpected) result shows up in your email...wait, I've just described Fiverr.

I'm not going to talk about the copyright issue at the moment. I am just dealing with the "but is it art, and did you really make it or did you just press the button?" aspect. There's a dirty secret (well, not secret, but often dirty) to popular AI art, and that there is a hell of a lot of repainting. Basically, if you see spectacular AI images of the "trending on Artstation" variety, it is because someone with good PhotoShop skills spent the same annoying finicky hours grinding away with the little brushes and the blur tool and all that.

The difference is really where the effort needs to be. Take blending. Blending is a huge pain in the butt in any media. Lots of dabbing the brush in multiple slight variations of the same color and using just the right amount of water and going back and back and back and back.

The same thing can be "...easily accomplished by a computer." In the right conditions. With the older tools, like Gaussian blur, it is a one-step; use the selection, run math on it. AI does convergence towards goal, meaning it will blend properly in the ugly edge area.

It can use the same understanding-by-example to add sheen or dew droplets or bokeh or all sorts of things that are really, really fiddly to do without it.

What is lacks is any constructivist understanding, It can pick up rules based on what was in the images it was trained on. That's it. It "knows" that nuts are found near bolts. It doesn't understand that nuts go on bolts, and that they don't work sideways, either.

So for AI generation of fashion, it gets scary in the way it can put hyper-detailed wrinkles and stitching and seams and all that. But it has no understanding of how clothes are constructed. It can do a perfect buckle, but it has no function (well...maybe it was trained on Liedfeld.)

I guess it is a perfect mix for bad steampunk. "Just glue some gears on it."

Sorry...meant to write a post about process and lessons learned, but the preamble went really long.

Dancing about architecture

That is actually the state-of-art of AI right now. It will get better, and quickly. Those hard-learned skills people are using today will pass out of use as the software gets over certain hurdles.

Some things are much harder, though, and there will always be this frustrating indirect aspect to it. That's why I said dancing about architecture; but this is more like trying to communicate a building design through interpretive dance. Or chanting memorized spells that worked for someone else and worked last week and you almost, but not quite, understand why they work at all...

So first there's the learning curve. As with every field, it is a whole complicated world and it is hard to get over that first hurdle and orient to the ground and start understanding what the processes are, what the issues are.

I'm currently running Stable Diffusion -- which is actually a model trained on the LAION-5B database, through the AUTOMATIC1111 front end, which is a webgui that, like the Stable Diffusion package is running on Python (including torch, which pipes to cuda, and xformers, which I currently have disabled (because honestly my python-fu is weak and I needed to invoke the arg "--lowvram" due to my not-quite-up-to-all-that-parallel-processing graphics card.)

So, yeah, command-line stuff, installing off github, all that good stuff. And a starter checkpoint model -- the usual v1.5, and I don't actually remember what flavor, although I have four other checkpoints and a handful of lora and even a textual inversion or so in my folder now.

But mostly I'm messing around with upscaling, trying out the difference between, say, ERSGAN_4x and the R-ESRGANs, and trying to remember to get a newer VAE, and what is all this about ControlNet and Openpose?

But what does this all mean?

So the current state of the art of a theoretical AI pipeline is not really "push the button get art." More like "study the magic words huddled alone in a garret, sometimes they work, sometimes demons pop out." I had one of those weird demons...I got a sort of totem pole of Michelle Trachtenberg (yes -- celebrity names are one of the big Invokes in the Deep Magic of chanting Prompts towards the keyboard. 

Here's PART of one found in the wild: ..glamorous pose, trending on ArtStation, dramatic lighting, ice, fire and smoke, orthodox symbolism Diesel punk, mist, ambient occlusion, volumetric lighting...

And it gets even more fun when you start seeing modifiers. [jennifer connely:0.8 jennifer anniston],(((long hair))), (bokeh)...

Yeah, what did I say about demonology? The engine goes crazy on celebrities. Even more so artists. I was trying out text to image and I put a "william turner" in there and I got the most lovely clouds and dramatic lights...and the damned Fighting Temeraire hovering around the cornfields, refusing to be banished.

Because that leads to why this is arcane. We don't know what the machine is thinking. We can guess. Put "steve jobs" in there and an apple logo will show up like a buddy, because (like turner) the one is associated with the other with the majority of materials the model will have been trained on.

It doesn't understand the world. Of course. It is the basic problem of AI. Plus, the things that look difficult for us...well, as pioneer Hans Moravec put it, a computer can beat a man at chess. A five-year-old kid can pick up a toppled king and put it back on the board.

Machine vision is making progress but it is brute force stuff. People lie awake at nights wondering what we are actually teaching the self-driving cars to avoid. (This goes way back... a wartime story that might be a tall tale of dogs trained to run under tanks. On the day of the battle, they had bombs strapped to their backs. Unfortunately in all the noise and confusion they vastly preferred to run under the tanks they'd been training on...)

We can't talk to the machines. Not the Stable Diffusion trained checkpoints, not Google search engine, not Amazon's algorithm. We can only make educated guesses about what the world looks like to it. You might be trying to render "elephant in a church" and you keep getting Roman soldiers. Why? Oh turns out one person wrote a prompt to reconstruct a campaign against Carthage for a personal render, and it ended up on a popular website with beginners, and there are only three elephants in the images shown to the v.1 engine, so it always assumes you are going to want Romans, too.

Totally made up. Actually, the AI is worse than Doctor Brundle's teleporter. It isn't even certain how many heads one person has (it can never get fingers right) so with two people in there...weird things happen.

And it is the text-to-image that is the current hot button. The idea on the one hand that someone could skip all those years in art school. The idea on the other hand that you are stealing J.W.R. Turner's wonderful clouds. And worse, the internet has fads, and the AI world is practically a world of script-kiddies, borrowing arcane words of power from the old masters and using them with abandon. So one person discovers that "Sarah Andersen" is as powerful a cry as calling on the All-Seeing Eye of Agamoto and pretty soon half the user base is doing it to crank out that image flavor of the week.

But upscaling -- the reason, honestly, I looked at this stuff in the first place -- is leveraging many of the same tools and assets. You upscale a brick wall and the AI is not just using the photograph you took and the PhotoShop hand painting you did and the freely-given math and programming the whole edifice rests on, but it is going through a stash of images from, well, Getty among others.

And they are probably not recognizable, not in this use. And I have both sympathy and schadenfreude, because Getty charges hard for the use of the materials they have there. And this is in the end probably not a war that anyone will win. Like the fight against mix tapes, which became music sharing, which became large-scale piracy...and the music industry still is not back in control like they want to be (and never will).

But the moral aspect is still back there.

And there is the practical problem. I am on the fence whether it "counts" as art if you are still having to, basically, work just as hard. Are we going to tell people they aren't real artists, real musicians, real writers, because they lack innate talent and had to slog through the trenches instead?

My small encounter with the stuff is that at least some of the (few) art skills I came by -- via that same long round of study and practice -- are still useful here. Half of it is understanding the world well enough to understand what the AI might be looking at. Those skills of being able to both abstract the idea and be specific about the details of a cat or a building or a face. And the AI does share some of that language; if you note, the terms photographers developed to talk about their approaches are recognized by the AI and used by the prompt-chanters.

Another is that repainting is the hidden skill. I don't think more than a handful of people are at the leading edge of popularity in AI art without being able to work PhotoShop themselves.

As I mentioned, the AI can't be relied on to count heads, know how hems work, and hands are a nightmare creation. Far too often, the cycle is actually to get close, then to drop into a paint program and throw a dash of corrective paint on that blot or scar the AI has gotten all worked up about and insists on re-working into a miniature cathedral flanked by grotesques.

This is where both sides of that artistic training come into play, as you can lead the AI in the directions you'd rather it went, with a little understanding and artistic skill and a good hand on the brush.

And that's all I'm going to say at the moment. Sometimes a Fox takes place in 2019 and AI hasn't blown up big, not yet, not for them. And it is also just a bit too bleeding-edge for a wider readership. Sure, I might slip a reference to "Team Catradora" in here and there, but there was a reason why Penny was still using Facebook...

Waiting For Goto

 Seems like a lot of my life is spent waiting while a computer does something. 3D is terrible for that; Bryce3D was infamous for the long render times.

Compiling, even. Downloads, especially. I have a Windows PC for playing games and it runs Steam and it is pretty much a guarantee that every time I want to play a quick bit of some game I have to wait 5-10 minutes for the Steam Update to do whatever it is doing.

I just bought a new machine so I can set up the dual monitors I want. Oddly enough, for writing more than anything else, but it will certainly help with PhotoShop as well. When that shows up, there's going to be a long fun time of re-downloading all the various software I need. That, and somehow an OS straight from the manufacturer needs hours of patches loaded on it first thing.

Oh, and I'm messing around with Stable Diffusion now. Trying to see if it can do some upscaling/inpainting for my book cover woes. (I'm also waiting on a new cover artist -- but that's waiting on humans. At least I think they are humans. For all I know, they are over there waiting on their AI program to finish a render...)