Alma Katsu on Thrillers, Deepfakes, and Keeping Up with New Technology

My desire to write Incarnate, a novel about the dangers of technology, came from my time as a technology futurist for CIA. Being someone who assessed where new technologies might take us meant drinking daily from a firehose of information: staying on top of technology news, of course, but also mining private sources, talking with experts, researchers, and folks at start-ups.

Every new technology goes through the same cycle, to varying degrees, of boom and leveling out (or crashing), but with generative AI (now just referred to as AI) the boom was exponential, faster than any technology I’d followed in my years in the business. I continued as a consultant after retiring from government but, as genAI caught fire, the pace was too much. I couldn’t continue and grow my writing career. So, I quit the foresight business, thinking I was getting off the hamster wheel for good.

Then came the idea for Incarnate.

My new novel is about the unintended consequences of technology. Spend any time in technology forecasting and you soon learn that unintended consequences are what you need to watch out for—while being, at the same time, nearly impossible to predict.

Where do unintended consequences come from? Typically, the creator of a technology foresees a specific application for their product. If a product is lucky enough to be widely adopted, now thousands of people are thinking up new applications for it—and that’s when those unintended consequences rear their petulant heads.

The smartphone is the textbook example: while Apple had envisioned an app store so outside developers could market their extensions of the phone’s functionality—like putting an extra blade on a Swiss Army knife—no one foresaw the tsunami of apps that would eventually change the way we manage information, even the way we think, and make the smart phone the indispensable tool it is today.

The idea for Incarnate came from an urge to comment on the unintended consequences of social media and deep fakes. In case you’re unfamiliar with the term, “deepfake” referred to the use of a combination of computer programs to create an image or video of something that never happened. In the beginning, these were videos of politicians or celebrities saying or doing something that they never did.

In intelligence, we saw the threat from deep fakes being primarily from bad actors who would use them to convince the masses that their leader said or did something that could be destabilizing. Fake news may have been common at the time, but being able to produce a convincing video spouting disinformation was still difficult and required special skills. But now, using genAI, anyone can create what is essentially a “deepfake” with a free online tool. People create them to prank their friends, exact revenge on ex-lovers, or just to alleviate boredom.

As I was wrapping up revisions, a real-life development in genAI threatened to upend the novel. The premise hinges on Dorothy, the main character, having the unique ability to basically create magic with her programming. It was right around this time that OpenAI’s SoraAI burst into public discourse.

Sora was a chatbot that could create video and audio from whole cloth, based on prompts—ChatGPT for video, basically. Sora came on so fast and so strong that there was speculation in Hollywood that we’d soon see widespread use of AI in films and television, bolstered by a $1 billion investment deal between Disney and OpenAI. It seemed like the beginning of the end for video creators. It was an existential threat to Dorothy—and my story.

At this point, we were still over a year out from publication. Chances were good that SoraAI would be even more commonplace by then and the story of Hollywood’s decimated would be well known. I couldn’t ignore it—it would seem like a big, gaping oversight to readers. But that meant I had to figure out how to neutralize it but in a wholly plausible way.

Luckily, that’s another aspect of forecasting—part of the job is coming up with possible scenarios for how the issue might play out.

First, I acknowledged the existence of SoraAI in my story by creating a doppelganger: SonaAI. I put it in the world of my story, and I had my characters in the visual effects industry worried it would take their jobs away.

But I knew, from experience, that the timing of a technological takeover is not a sure thing. It’s subject to a wide variety of factors, most of which are out of a developer’s control. So, I decided to roll the dice. In my story, SonaAI would still be on the rise but that it would at a more realistic pace, not meteoric. Dorothy and her colleagues were constantly asked by clients of they were afraid that SonaAI was going to take away their livelihood and, of course, they were.

But I kept the wolf at the door. I didn’t make Sora/SonaAI the monster. (That’s another story.)

To me, the monster of the story is human weakness. All-too-human greed. The point in a novel is the story. Incarnate isn’t a course in machine learning: if anything, it’s a look into the liminal space between the stark, cold reality of science and the magic of the unknown. For most people, AI is magic. To explain it in too much detail eats away that magic and ends up blunting the point I was hoping to make: beware embracing the unknown, whether it’s a spiritual demon or a technology made for deception.

Then, a miracle happened. In a crazy, surprising twist, OpenAI announced just a few months after the Disney deal that it was shutting down its Sora app for consumers.

This meant my readers would not go into Incarnate knowing that the visual effects industry had been decimated. That the world I’d painted was not already obsolete.

I’d rolled the dice and won. It felt like the tech gods (or the demons in the machines) were looking out for me.

And, if you’re looking for an answer as to why, if Sora was so popular, OpenAI decided to shut it down: The most logical reason is that it was too expensive to support, preventing OpenAI from investing in other genAI applications that, in the long, made more business sense. It seemed like an impossible outcome at the time, SoraAI’s success predicted by tech and business experts, but such about-faces do happen in the tech world—it’s just that they’re rare.

***